Method, apparatus and program products for retrieving moving image

ABSTRACT

An apparatus for retrieving moving image according to the present invention has an extracting unit for extracting the information of the feature quantity of the moving image from the fixed moving image data, an extracting unit for extracting the information of the unique identification of the moving image to identify the moving image according to the information of the feature quantity of the moving image, and a retrieving unit for retrieving and identifying the fixed moving image from the object moving image for retrieving by comparing the information of the unique identification of the moving image with the information of the unique identification of the moving image beforehand extracted from the object moving image. Therefore, the information of the unique identification of the moving image for retrieving the moving image depends on the content of the moving image can be extracted, consequently, the identical moving image can be retrieved and identified from the moving image by using the information of the unique identification of the moving image at high speed.

FIELD OF THE INVENTION

This invention relates to a method and an apparatus for retrievingmoving image, which retrieve and identify fixed moving image from somemoving image. And more particularly to, a method and an apparatus forretrieving moving image, in which the information of the uniqueidentification of moving image which depends on the content of themoving image is extracted and registered, the registered moving image isretrieved and identified earlier than video rate from moving image suchas moving image archives sent in broadcasting and the like.

BACKGROUND OF THE INVENTION

It is well known that there are various technologies on theinternational standard encoding system such as MPEG (Motion PictureExperts Group) as the information compressing technology for storing andtransmitting moving image. It becomes possible that moving image issufficiently handled even in the PC (Personal Computer), if the encodedmoving image compressed by this international standard encoding systemis used. The technology for monitoring the specific moving image byusing the PC, for example monitoring a television commercial(identifying moving image), is ever studied and is developed.

As the conventional method for retrieving moving image, there areseveral methods for retrieving moving image, for example a method usingthe keyword retrieval, a method using the visual discrimination by thevisual observation of the user, and a method using the number ofcontinuous frames on the shot of moving image disclosed in JapanesePatent Publication No. JP7-retrieving moving image by detecting the cutpoint of the moving image. In this apparatus for retrieving the cutpoint of moving image disclosed in Japanese Patent Publication No.JP7-236153, the time variation of luminance difference between frames ofthe reduction image which is made from prepared reduction image and themean value component data of the information of compressed and encodedmoving image and the time variation of color difference histogramcorrelation value are used in order to obtain a cut point at high speedfrom moving image.

And, there is an apparatus for processing moving image disclosed inJapanese Patent Publication No. JP7-152779 as an apparatus for detectingthe characteristic frame of moving image. In this apparatus forprocessing moving image disclosed in Japanese Patent Publication No.JP7-152779, the low frequency part of DCT (Discrete Cosine Transform)component is used in order to detect characteristic frame that is usedfor as index information (retrieving information), for example a cutpoint frame, a motionless frame, a character frame, a specific colorframe, etc., from compressed moving image data including the DCToperation.

In the conventional methods and apparatuses for retrieving moving image,however, there are disadvantages as follows.

That is to say, in the conventional method for retrieving moving imageby the keyword retrieval, there is a disadvantage that the considerablecost for registering keyword index information is taken, because tobeforehand register the keyword index information is necessary. Andmore, there is a disadvantage with the case in which keyword indexinformation differs even in the case of retrieving of identical movingimage by the user, because the feeling of moving image of each user whoretrieves moving image and the way of understanding are different.Moreover, there is a disadvantage that the time for choosing the keywordindex of the desire is taken, even the system for inputting keywordindex in which the user who retrieves moving image from keyword indexinformation presented by a list type is made to choose the keyword indexof the desire.

And, in the conventional method for retrieving moving image using thevisual discrimination by the visual observation of the user, there is adisadvantage that hands are taken and the cost is high, because it isthe method in which the user who retrieves moving image memorizes movingimage that the user intends to retrieve and identify, and the userretrieves and identifies the moving image of the desire by observing allmoving images.

And, in the method for retrieving moving image using the number ofcontinuous frames on the shot which constitutes moving image disclosedin Japanese Patent Publication JP7-46517, there is a disadvantage thatit might be misunderstood that moving image is identical with movingimage for retrieving, if the number of continuous frames of the movingimage agrees even if the content of the moving image differs from themoving image for retrieving, because the number of continuous frames isnot index information reflecting the content of the meaning of movingimage.

And, in the conventional real time discrimination method of image scenebased on the time series coding of cutting, there is a disadvantage thatthe time for the extracting process of a cut point frame is taken andthe cost also rises, because it is necessary to accurately extract a cutpoint frame in order to obtain a right collation result.

And, in the apparatus for retrieving the cut point of moving imagedisclosed in Japanese Patent Publication No. JP7-236153, there is adisadvantage that it is impossible to retrieve and identify moving imageof the desire automatically, because the detection of a cut point inmoving image is a purpose, and it is only to detect a cut point byregarding the time variation of luminance difference and the timevariation of color difference histogram correlation value.

In the apparatus for processing moving image disclosed in JapanesePatent Publication No. JP7-152779, there is a disadvantage that time istaken due to the complicated process and the cost is high. Because theinterim result which corresponds to the result of DCT operation by theprocess for reconstructing image data from compressed image data is madeto be signals of the information of the feature quantity of moving imageindex, at least a kind of moving image index is detected from thissignals of the information of the feature quantity of moving imageindex, and the sum of difference value with the moving image index ofexamined image is processed at the threshold, and then moving image isdetected as index image by the result of this threshold processing.

SUMMARY OF THE INVENTION

Accordingly, it is an object of the invention to provide a method and anapparatus for retrieving moving image, in which the information of theunique identification of moving image used for retrieving in accordancewith the content of the moving image can be automatically extracted evenin the computer with small resources such as PC, and in which movingimage can be retrieved and identified automatically by using theinformation of the unique identification of moving image.

According to the first feature of the invention, an apparatus forretrieving moving image to retrieve pre-determined moving image fromobject moving image, comprises first extracting means for extracting theinformation of the feature quantity of each frame from thepre-determined moving image, second extracting means for extracting theinformation of the unique identification of moving image to identify themoving image in accordance with the information of the feature quantity,extracted by the first extracting means, and retrieving means forretrieving the pre-determined moving image from the object moving imageby comparing the information of the unique identification, extracted bythe second extracting means, with the information of the uniqueidentification prepared in accordance with the object moving image.

According to the second feature of the invention, a method forretrieving moving image to retrieve pre-determined moving image fromobject moving image, comprises the steps of (A) extracting theinformation of the feature quantity of each frame from thepre-determined moving image, (B) extracting the information of theunique identification of moving image to identify the moving image inaccordance with the information of the feature quantity, extracted bysaid step (A), and (C) retrieving the pre-determined moving image fromthe object moving image by comparing the information of the uniqueidentification, extracted by the step (B), with the information of theunique identification prepared in accordance with the object movingimage.

According to the third feature of the invention, a program forretrieving moving image to retrieve predetermined moving image fromobject moving image, which is executed by computer system, comprises thesteps of (A) extracting the information of the feature quantity of eachframe from the pre-determined moving image, (B) extracting theinformation of the unique identification of moving image to identify themoving image in accordance with the information of the feature quantity,extracted by the step (A), and (C) retrieving the pre-determined movingimage from the object moving image by comparing the information of theunique identification, extracted by the step (B), with the informationof the unique identification prepared in accordance with the objectmoving image.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described in more detail in conjunction with theappended drawings, wherein:

FIG. 1 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 2A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 2B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 3 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 4A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 4B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 5 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 6A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 6B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 7 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 8A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 8B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 9 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 10A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 10B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 11 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 12A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 12B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 13 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 14 is a conceptual view showing the process of DCT (discrete cosinetransform) operation in MPEG;

FIG. 15A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 15B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 16 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 17A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 17B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 18 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 19A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 19B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 20 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 21A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 21B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 22 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 23A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 23B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 24 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 25A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 25B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 26 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 27A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 27B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 28 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 29A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 29B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 30 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 31A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 31B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 32 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 33A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 33B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 34 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 35A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 35B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 36 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 37A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 37B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 38 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 39A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 39B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 40 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 41A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 41B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 42 is a block diagram showing the apparatus for retrieving movingimage according to the present invention;

FIG. 43A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 43B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 44 is a block diagram showing the apparatus for retrieving movingimage according to the present invention; and

FIG. 45A is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention.

FIG. 45B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

FIG. 46 is a block diagram showing the apparatus for monitoringtelevision commercials according to the present invention; and

FIG. 47A is a flowchart of the process in the apparatus for monitoringtelevision commercials according to the present invention.

FIG. 47B is a flowchart of the process in the apparatus for retrievingmoving image according to the present invention;

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following, a method and an apparatus for retrieving moving image,and a recording medium according to the invention are explained indetails referring to the drawings.

An apparatus and a method for retrieving moving image in the firstpreferred embodiment according to the invention will be explained inFIGS. 1, 2A and 2B.

FIG. 1 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. In FIG. 1, the apparatus forretrieving moving image comprises an extracting unit 102 for extractinginformation 103 of the feature quantity of moving image, which includesphysical information of the feature quantity of moving image such asinformation of luminance and color difference (Y, Cb, Cr) of each frameand/or information of color (R, G, B) of each frame, from moving imagedata 101, an extracting unit 104 for arranging at least one physicalinformation included in the information 103 (such as the information ofthe luminance and the color difference (Y, Cb, Cr) of each frame and/orthe information of the color (R, G, B) of each frame) on the time axisand for extracting information 105 of the unique identification ofmoving image to identify the moving image from the arrangement of thephysical information on the time axis, a storing unit 106 for storingthe information 105 of the unique identification of the moving imageextracted by the extracting unit 104, a retrieving unit 109 forretrieving identical information with the information 105 of the uniqueidentification of the moving image stored in the storing unit 106 frominformation 108 of the unique identification of moving image beforehandextracted from other moving image data, a storing unit 110 for storingthe retrieving result provided by the retrieving unit 109, and an outputunit 111 for displaying and/or printing the retrieving result stored inthe storing unit 110. On the apparatus for retrieving the moving imageaccording to the present invention constituted like the above, anoperation will be explained in the following.

FIGS. 2A and 2B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 1. In FIGS. 1, 2A and 2B, at first the extracting unit 102 forextracting the information of the feature quantity of the moving imagefetches the moving image data 101 to be a retrieval key for identifyingthe moving image (in the step 201). Next, the extracting unit 102extracts the information 103 of the feature quantity of the movingimage, which includes the physical information of the feature quantityof the moving image such as the information of the luminance and thecolor difference (Y, Cb, Cr) of each frame and/or the information of thecolor (R, G, B) of each frame, from the moving image data 101 (in thestep 202).

Next, the extracting unit 104 for extracting the information of theunique identification of the moving image arranges at least one valuefrom values of the physical information of the feature quantity of themoving image (such as the information of the luminance and the colordifference (Y, Cb, Cr) of each frame and/or the information of the color(R, G, B) of each frame) based on the information 103, mean value of thevalues, total value of the values and/or difference value of the valueson the time axis. Then, the extracting unit 104 picks out all valuesarranged on the time axis, or some values from the values arranged onthe time axis at uniform intervals or irregular intervals. And then, theextracting unit 104 extracts the picked out values as the information105 of the unique identification of the moving image to be used foridentifying the moving image (in the step 203).

The storing unit 106 for storing the information of the uniqueidentification of the moving image stores the information 105 of theunique identification of the moving image extracted by the extractingunit 104 (in the step 204).

On the other hand, the information 108 of the unique identification ofthe moving image is extracted from other moving image data (the objectmoving image data for retrieving) (in the step 205). This extractingprocess can be executed by the processing equal to the foregoing steps201 to 204.

Then, the retrieving unit 109 for retrieving the information of theunique identification of the moving image compares the information 105of the unique identification of the moving image stored in the storingunit 106 with the information 108 of the unique identification of themoving image extracted from the other moving image data (in the step206).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 206 to 209 is executed for allof the information 108 of the unique identification of the moving imageextracted from the other moving image data (in the step 210).

The apparatus and the method for retrieving the moving image accordingto the invention are explained the above. The above mentioned extractingunit 104 arranges at least one value from values of the physicalinformation of the feature quantity of the moving image (such as theinformation of the luminance and the color difference (Y, Cb, Cr) ofeach frame and the information of the color (R, G, B) of each frame)based on the information 103 of the feature quantity of the movingimage, mean value of the values, total value of the values and/ordifference value of the values on the time axis, and picks out allvalues arranged on the time axis, or some values from the valuesarranged on the time axis at uniform intervals or irregular intervals,and then extracts the picked out values as the information 105 of theunique identification of the moving image to be used for identifying themoving image. However, the above mentioned extracting unit 104 mayarrange at least one value from values of a part or multiple parts ofthe physical information of the feature quantity of the moving image(such as the information of the luminance and the color difference (Y,Cb, Cr) of each frame and the information of the color (R, G, B) of eachframe) based on the information 103 of the feature quantity of themoving image, mean value of the values, total value of the values and/ordifference value of the values on the time axis, and pick out all valuesarranged on the time axis, or some values from the values arranged onthe time axis at uniform intervals or irregular intervals, and thenextract the picked out values as the information 105 of the uniqueidentification of the moving image to be used for identifying the movingimage. That is to say, values of information of feature quantity ofmoving image in each block unit may be calculated by dividing a frameinto several blocks, and the information 105 of the uniqueidentification of the moving image may be extracted by using all of theabove mentioned values of the information of the feature quantity of themoving image. And also, the information 105 of the unique identificationof the moving image may be extracted by using some values of theinformation of the feature quantity of the moving image. As suchembodiment, for example, values of the information of the featurequantity of the moving image, which are calculated from blocks that arelocated on the part except for the part of time display and the part ofaround frame (the part with unevenness), are used.

And more, the extracting unit 104 may calculate values after executingthe weighting on the physical information of the feature quantity of themoving image (such as the information of the luminance and the colordifference (Y, Cb, Cr) of each frame and/or the information of the color(R, G, B) of each frame) based on the information 103, then may applythe values, mean value of the values, total value of the values and/ordifference value of the values.

Moreover, the extracting unit 104 may calculate values of the physicalinformation of the feature quantity of the moving image, mean value ofthe values, total value of the values and/or difference value of thevalues, and then may quantize the calculated values.

As mentioned above, according to the first embodiment of the invention,information of the unique identification of moving image for retrievingmoving image depends on the content of the moving image can beextracted, consequently, identical moving image can be retrieved andidentified from the moving image by using the information of the uniqueidentification of the moving image at high speed.

An apparatus and a method for retrieving moving image in the secondpreferred embodiment according to the invention will be explained inFIGS. 3, 4A and 4B.

FIG. 3 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 3 withFIG. 1. In FIG. 3, the apparatus for retrieving moving image comprisesan extracting unit 302 for extracting information 303 of the featurequantity of compressed moving image, which includes DCT (Discrete CosineTransform) coefficients of each frame and information of motioncompensation of each frame, from compressed moving image data 301provided by the standard compression encoding process for moving imagesuch as MPEG, an extracting unit 304 for arranging at least oneinformation of feature quantity from the DCT coefficients of each frameand the information of the motion compensation of each frame, which areincluded in the information 303, on the time axis and for extractinginformation 305 of the unique identification of compressed moving imageto identify the compressed moving image from the arrangement of theinformation of the feature quantity on the time axis, a storing unit 306for storing the information 305 of the unique identification of thecompressed moving image extracted by the extracting unit 304, aretrieving unit 309 for retrieving identical information with theinformation 305 of the unique identification of the compressed movingimage stored in the storing unit 306 from information 308 of the uniqueidentification of compressed moving image beforehand extracted fromother moving image data, a storing unit 110 for storing the retrievingresult provided by the retrieving unit 309, and an output unit 111 fordisplaying and/or printing the retrieving result stored in the storingunit 110. On the apparatus for retrieving the moving image according tothe present invention constituted like the above, an operation will beexplained in the following.

FIGS. 4A and 4B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 3. Still, an identical step number is appended to the stephaving the identical process in FIGS. 4A and 4B with FIGS. 2A and 2B. InFIGS. 3, 4A and 4B, at first the extracting unit 302 for extracting theinformation of the feature quantity of the compressed moving imagefetches the compressed moving image data 301 to be a retrieval key foridentifying the compressed moving image (in the step 401). Next, theextracting unit 302 extracts the information 303 of the feature quantityof the compressed moving image, which includes the DCT coefficients ofeach frame and the information of the motion compensation of each frame,from the compressed moving image data 301 (in the step 402).

Next, the extracting unit 304 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of the compressed movingimage from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the valuesand/or difference value of the values. On the other hand, the extractingunit 304 calculates values such as a motion vector, a mean motion vectorbetween frames before and behind, a total motion vector, a differencevector and/or a motion vector on a whole frame from the information ofthe motion compensation. And the extracting unit 304 arranges at leastone value from the above mentioned values on the time axis and thenpicks out all values arranged on the time axis, or some values from thevalues arranged on the time axis at uniform intervals or irregularintervals. And then, the extracting unit 304 extracts the picked outvalues as the information 305 of the unique identification of thecompressed moving image to be used for identifying the compressed movingimage (in the step 403).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 305of the unique identification of the compressed moving image extracted bythe extracting unit 304 (in the step 404).

On the other hand, the information 308 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 401 to 404.

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 308 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 308 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

The apparatus and the method for retrieving the moving image accordingto the invention are explained the above. The above mentioned extractingunit 304 calculates values of the physical information of the featurequantity of the compressed moving image from the DCT coefficients ofeach frame, which is included in the information 303, mean value of thevalues, total value of the values and/or difference value of the values,and values such as a motion vector, a mean motion vector between framesbefore and behind, a total motion vector, a difference vector and/or amotion vector on the whole frame from the information of the motioncompensation. And the above mentioned extracting unit 304 arranges atleast one value from the above mentioned values and then picks out allvalues arranged on the time axis, or some values from the valuesarranged on the time axis at uniform intervals or irregular intervals.And then, the above mentioned extracting unit 304 extracts the pickedout values as the information 305 of the unique identification of thecompressed moving image to be used for identifying the compressed movingimage. However, the above mentioned extracting unit 304 may calculatevalues of a part or multiple parts of the physical information of thefeature quantity of the compressed moving image from the DCTcoefficients of each frame, which is included in the information 303,mean value of the values, total value of the values and/or differencevalue of the values, and values such as a motion vector, a mean motionvector between frames before and behind, a total motion vector, adifference vector and/or a motion vector on the whole frame from a partor multiple parts of the information of the motion compensation. That isto say, values of information of feature quantity of moving image ineach block unit may be calculated by dividing a frame into severalblocks, and the information 305 of the unique identification of themoving image may be extracted by using all of the above mentioned valuesof the information of the feature quantity of the moving image. Andalso, the information 305 of the unique identification of the movingimage may be extracted by using some values of the information of thefeature quantity of the moving image. As such embodiment, for example,values of the information of the feature quantity of the moving image,which are calculated from blocks that are located on the part except forthe part of time display and the part of around frame (the part withunevenness), are used.

And more, the extracting unit 304 may execute the weighting on thephysical information of the feature quantity of the compressed movingimage from the DCT coefficients of each frame, which is included in theinformation 303, then may calculate values from the weighted informationof the feature quantity of the compressed moving image, mean value ofthe values, total value of the values and/or difference value of thevalues. On the other hand, the extracting unit 304 may execute theweighting on the information of the motion compensation and maycalculate a motion vector, a mean motion vector between frames beforeand behind, a total motion vector, a difference vector and/or a motionvector on a whole frame from the weighted information of the motioncompensation.

Moreover, in the extracting unit 304, values provided by quantizing theabove mentioned values of the physical information of the featurequantity of the compressed moving image and the like, and valuesprovided by quantizing the above mentioned motion vectors, and the likemay be applied.

As mentioned above, according to the second embodiment of the invention,information of the unique identification of moving image for retrievingmoving image depends on the content of the compressed moving imageprovided by the standard compression encoding process for moving imagesuch as MPEG can be extracted, consequently, identical moving image canbe retrieved and identified from the moving image by using theinformation of the unique identification of the moving image at highspeed.

An apparatus and a method for retrieving moving image in the thirdpreferred embodiment according to the invention will be explained inFIGS. 5, 6A and 6B.

FIG. 5 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 5 withFIG. 1. In FIG. 5, the apparatus for retrieving moving image comprisesan extracting unit 102 for extracting information 103 of the featurequantity of moving image, which includes physical information of thefeature quantity of moving image such as information of luminance andcolor difference (Y, Cb, Cr) of each frame and/or information of color(R, G, B) of each frame, from moving image data 101, a detecting unit501 for detecting a cut point of the moving image data 101 asinformation 502 of a cut point, an extracting unit 503 for arranging atleast one physical information included in the information 103 (such asthe information of the luminance and the color difference (Y, Cb, Cr) ofeach frame and/or the information of the color (R, G, B) of each frame)on the time axis and for extracting information 505 of the uniqueidentification of moving image to identify the moving image in each shotunit of the moving image data 101 divided by using the information 502of the cut point from the arrangement of the physical information on thetime axis, a storing unit 106 for storing the information 505 of theunique identification of the moving image extracted by the extractingunit 503, a retrieving unit 109 for retrieving identical informationwith the information 505 of the unique identification of the movingimage stored in the storing unit 106 from information 508 of the uniqueidentification of moving image beforehand extracted from other movingimage data, a storing unit 110 for storing the retrieving resultprovided by the retrieving unit 109, and an output unit 111 fordisplaying and/or printing the retrieving result stored in the storingunit 110. On the apparatus for retrieving the moving image according tothe present invention constituted like the above, an operation will beexplained in the following.

FIGS. 6A and 6B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 5. Still, an identical step number is appended to the stephaving the identical process in FIGS. 6A and 6B with FIGS. 2A and 2B. InFIGS. 5, 6A and 6B, at first the extracting unit 102 for extracting theinformation of the feature quantity of the moving image and thedetecting unit 501 for detecting the cut point fetch the moving imagedata 101 to be a retrieval key for identifying the moving image (in thestep 601).

Next, the extracting unit 102 extracts the information 103 of thefeature quantity of the moving image, which includes the physicalinformation of the feature quantity of the moving image such as theinformation of the luminance and the color difference (Y, Cb, Cr) ofeach frame and/or the information of the color (R, G, B) of each frame,from the moving image data 101 (in the step 202).

And, the detecting unit 501 detects the cut point of the moving imagedata 101 as the information 502 of the cut point (in the step 602). Thiscut point of the moving image data 101 shows a point where the scene ofthe moving image data 101 changes or a point where the camera forphotographing the moving image data 101 switches.

Next, the extracting unit 503 for extracting the information of theunique identification of the moving image arranges at least one valuefrom values of the physical information of the feature quantity of themoving image (such as the information of the luminance and the colordifference (Y, Cb, Cr) of each frame and/or the information of the color(R, G, B) of each frame) based on the information 103, mean value of thevalues, total value of the values and/or difference value of the valueson the time axis. Then, the extracting unit 503 picks out all valuesarranged on the time axis, or some values from the values arranged onthe time axis at uniform intervals or irregular intervals. And then, theextracting unit 503 extracts the picked out values in each shot unit ofthe moving image data 101 divided by the information 502 of the cutpoint as the information 505 of the unique identification of the movingimage to be used for identifying the moving image (in the step 603).

The storing unit 106 for storing the information of the uniqueidentification of the moving image stores the information 505 of theunique identification of the moving image extracted by the extractingunit 503 (in the step 604).

On the other hand, the information 508 of the unique identification ofthe moving image is extracted from other moving image data (the objectmoving image data for retrieving) (in the step 205). This extractingprocess can be executed by the processing equal to the foregoing steps601 to 604.

Then, the retrieving unit 109 for retrieving the information of theunique identification of the moving image compares the information 505of the unique identification of the moving image stored in the storingunit 106 with the information 508 of the unique identification of themoving image extracted from the other moving image data (in the step206).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 206 to 209 is executed for allof the information 508 of the unique identification of the moving imageextracted from the other moving image data (in the step 210).

The apparatus and the method for retrieving the moving image accordingto the invention are explained the above. The above mentioned extractingunit 503 for extracting the information of the unique identification ofthe moving image arranges at least one value from values of the physicalinformation of the feature quantity of the moving image (such as theinformation of the luminance and the color difference (Y, Cb, Cr) ofeach frame and/or the information of the color (R, G, B) of each frame)based on the information 103 of the feature quantity of the movingimage, mean value of the values, total value of the values and/ordifference value of the values on the time axis. However, the extractingunit 503 may arranges at least one value from values of a part ormultiple parts of the physical information of the feature quantity ofthe moving image (such as the information of the luminance and the colordifference (Y, Cb, Cr) of each frame and/or the information of the color(R, G, B) of each frame) based on the information 103 of the featurequantity of the moving image, mean value of the values, total value ofthe values and/or difference value of the values on the time axis. Thatis to say, values of information of feature quantity of moving image ineach block unit may be calculated by dividing a frame into severalblocks, and the information 505 of the unique identification of themoving image may be extracted by using all of the above mentioned valuesof the information of the feature quantity of the moving image. Andalso, the information 505 of the unique identification of the movingimage may be extracted by using some values of the information of thefeature quantity of the moving image. As such embodiment, for example,values of the information of the feature quantity of the moving image,which are calculated from blocks that are located on the part except forthe part of time display and the part of around frame (the part withunevenness), are used.

And more, the extracting unit 503 may calculate values after executingthe weighting on the physical information of the feature quantity of themoving image (such as the information of the luminance and the colordifference (Y, Cb, Cr) of each frame and/or the information of the color(R, G, B) of each frame) based on the information 103 of the featurequantity of the moving image, then may apply the values, mean value ofthe values, total value of the values and/or difference value of thevalues.

Moreover, the extracting unit 503 may calculate values of the physicalinformation of the feature quantity of the moving image, mean value ofthe values, total value of the values and/or difference value of thevalues, and then may quantize the calculated values.

As mentioned above, according to the third embodiment of the invention,the information 505 of the unique identification of the moving image forretrieving the moving image depends on the content of the moving imagecan be extracted by the less information quantity with utilizing thetime concept of a shot according to the information 502 of the cutpoint. And, identical moving image can be retrieved and identified fromthe moving image by using the information 505 of the uniqueidentification of the moving image at the higher speed.

An apparatus and a method for retrieving moving image in the fourthpreferred embodiment according to the invention will be explained inFIGS. 7, 8A and 8B.

FIG. 7 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 7 withFIG. 3. In FIG. 7, the apparatus for retrieving moving image comprisesan extracting unit 302 for extracting information 303 of the featurequantity of compressed moving image, which includes DCT (Discrete CosineTransform) coefficients of each frame and information of motioncompensation of each frame, from compressed moving image data 301provided by the standard compression encoding process for moving imagesuch as MPEG, a detecting unit 701 for detecting a cut point of thecompressed moving image data 301 as information 702 of a cut point, anextracting unit 703 for arranging at least one information of featurequantity from the DCT coefficients of each frame and the information ofthe motion compensation of each frame, which are included in theinformation 303, on the time axis and for extracting information 705 ofthe unique identification of compressed moving image to identify thecompressed moving image from the arrangement of the information of thefeature quantity on the time axis in each shot unit of the compressedmoving image data 301 divided by using the information 702 of the cutpoint, a storing unit 306 for storing the information 705 of the uniqueidentification of the compressed moving image extracted by theextracting unit 703, a retrieving unit 309 for retrieving identicalinformation with the information 705 of the unique identification of thecompressed moving image stored in the storing unit 306 from information708 of the unique identification of compressed moving image beforehandextracted from other moving image data, a storing unit 110 for storingthe retrieving result provided by the retrieving unit 309, and an outputunit 111 for displaying and/or printing the retrieving result stored inthe storing unit 110. On the apparatus for retrieving the moving imageaccording to the present invention constituted like the above, anoperation will be explained in the following.

FIGS. 8A and 8B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 7. Still, an identical step number is appended to the stephaving the identical process in FIGS. 8A and 8B with FIGS. 4A and 4B. InFIGS. 7, 8A and 8B, at first the extracting unit 302 for extracting theinformation of the feature quantity of the compressed moving image andthe detecting unit 701 for detecting the cut point fetch the compressedmoving image data 301 to be a retrieval key for identifying thecompressed moving image (in the step 801). Next, the extracting unit 302extracts the information 303 of the feature quantity of the compressedmoving image, which includes the DCT coefficients of each frame and theinformation of the motion compensation of each frame, from thecompressed moving image data 301 (in the step 402). And, the detectingunit 701 detects the cut point of the compressed moving image data 301as the information 702 of the cut point (in the step 802).

Next, the extracting unit 703 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of the compressed movingimage from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the valuesand/or difference value of the values. On the other hand, the extractingunit 703 calculates values such as a motion vector, a mean motion vectorbetween frames before and behind, a total motion vector, a differencevector and/or a motion vector on a whole frame from the information ofthe motion compensation. And the extracting unit 703 arranges at leastone value from the above mentioned values on the time axis and thenpicks out all values arranged on the time axis, or some values from thevalues arranged on the time axis at uniform intervals or irregularintervals. And then, the extracting unit 703 extracts the picked outvalues in each shot unit of the compressed moving image data 301 dividedby using the information 702 of the cut point as the information 705 ofthe unique identification of the compressed moving image to be used foridentifying the compressed moving image (in the step 803).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 705of the unique identification of the compressed moving image extracted bythe extracting unit 703 (in the step 804).

On the other hand, the information 708 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 801 to 804.

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 708 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 708 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

The apparatus and the method for retrieving the moving image accordingto the invention are explained the above. The above mentioned extractingunit 703 calculates values of the physical information of the featurequantity of the compressed moving image from the DCT coefficients ofeach frame, which is included in the information 303, mean value of thevalues, total value of the values and/or difference value of the values,and values such as a motion vector, a mean motion vector between framesbefore and behind, a total motion vector, a difference vector and/or amotion vector on a whole frame from the information of the motioncompensation. However, the extracting unit 703 may calculate values of apart or multiple parts of the physical information of the featurequantity of the compressed moving image from the DCT coefficients ofeach frame, which is included in the information 303, mean value of thevalues, total value of the values and/or difference value of the values,and values/such as motion vector, a mean motion vector between framesbefore and behind, a total motion vector, a difference vector and/or amotion vector on the whole frame from a part or multiple parts of theinformation of the motion compensation. That is to say, values ofinformation of feature quantity of moving image in each block unit maybe calculated by dividing a frame into several blocks, and theinformation 705 of the unique identification of the moving image may beextracted by using all of the above mentioned values of the informationof the feature quantity of the moving image. And also, the information705 of the unique identification of the moving image may be extracted byusing some values of the information of the feature quantity of themoving image. As such embodiment, for example, values of the informationof the feature quantity of the moving image, which are calculated fromblocks that are located on the part except for the part of time displayand the part of around frame (the part with unevenness), are used.

And more, the extracting unit 703 may execute the weighting on thephysical information of the feature quantity of the compressed movingimage from the DCT coefficients of each frame, which is included in theinformation 303, then may calculate the above mentioned values from theweighted information of the feature quantity of the compressed movingimage. On the other hand, the extracting unit 703 may execute theweighting on the information of the motion compensation and maycalculate the above mentioned motion vectors from the weightedinformation of the motion compensation.

Moreover, in the extracting unit 703, values provided by quantizing theabove mentioned values of the physical information of the featurequantity of the compressed moving image and values provided byquantizing the above mentioned motion vectors, and the like may beapplied.

As mentioned above, according to the fourth embodiment of the invention,the information 705 of the unique identification of the compressedmoving image for retrieving moving image depends on the content of thecompressed moving image provided by the standard compression encodingprocess for moving image such as MPEG can be extracted by the lessinformation quantity with utilizing the time concept of the shotaccording to the information 702 of the cut point. And, identical movingimage can be retrieved and identified from the moving image by using theinformation 705 of the unique identification of the compressed movingimage at the higher speed.

An apparatus and a method for retrieving moving image in the fifthpreferred embodiment according to the invention will be explained inFIGS. 9, 10A and 10B.

FIG. 9 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 9 withFIG. 1 and FIG. 3. In FIG. 9, the apparatus for retrieving moving imagecomprises an extracting unit 901 for extracting information of motioncompensation in the M×N (M and N are natural numbers) pixels unit ofmoving image data 101, a DCT (Discrete Cosine Transform) operation unit902 for obtaining a DCT coefficient by executing a DCT operation in theM′×N′ (M′ and N′ are natural numbers) pixels unit of the moving imagedata 101, an extracting unit 302 for extracting information 303 of thefeature quantity of compressed moving image, which includes DCTcoefficients of each frame and information of motion compensation ofeach frame, from compressed moving image data 301 provided by thestandard compression encoding process for moving image such as MPEG, anextracting unit 304 for arranging at least one information of featurequantity from the DCT coefficients of each frame and the information ofthe motion compensation of each frame, which are the information 303extracted by the extracting unit 302, or by the extracting unit 901 andthe DCT operation unit 902, on the time axis and for extractinginformation 305 of the unique identification of compressed moving imageto identify the compressed moving image from the arrangement of theinformation of feature quantity on the time axis, a storing unit 306 forstoring the information 305 of the unique identification of thecompressed moving image extracted by the extracting unit 304, aretrieving unit 309 for retrieving identical information with theinformation 305 of the unique identification of the compressed movingimage stored in the storing unit 306 from information 308 of the uniqueidentification of compressed moving image beforehand extracted fromother moving image data, a storing unit 110 for storing the retrievingresult provided by the retrieving unit 309, and an output unit 111 fordisplaying and/or printing the retrieving result stored in the storingunit 110. On the apparatus for retrieving the moving image according tothe present invention constituted like the above, an operation will beexplained in the following.

FIGS. 10A and 10B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 9. Still, an identical step number is appended to the stephaving the identical process in FIGS. 10A and 10B with FIGS. 4A and 4B.In FIGS. 9, 10A and 10B, at first whether inputted moving image data iscompressed is judged. (in the step 1001).

In case the inputted moving image data is the uncompressed moving imagedata 101, the extracting unit 901 for extracting the information of themotion compensation and the DCT operation unit 902 fetch the movingimage data 101 to be a retrieval key for identifying the moving image(in the step 1002).

The extracting unit 901 divides each frame of the moving image data 101into the M×N (M and N are natural numbers) pixels units, then calculatesthe information of the motion compensation in the M×N pixels unit (inthe step 1003). On the other hand, the DCT operation unit 902 divideseach frame of the moving image data 101 into the M′×N′ (M′ and N′ arenatural numbers) pixels units, then calculates the DCT coefficient byexecuting the DCT operation in the M′×N′ pixels unit (in the step 1004).

The information of the motion compensation in the M×N pixels unitprovided by the extracting unit 901 and the DCT coefficient in the M′×N′pixels unit provided by the DCT operation unit 902 are sent to theextracting unit 304 for extracting the information of the uniqueidentification of the compressed moving image as the information 303 ofthe feature quantity of the compressed moving image.

On the other hand, in case the inputted moving image data is thecompressed moving image data 301, the extracting unit 302 for extractingthe information of the feature quantity of the compressed moving imagefetches the compressed moving image data 301 to be a retrieval key foridentifying the compressed moving image (in the step 401). And theextracting unit 302 extracts the information 303 of the feature quantityof the compressed moving image, which includes the DCT coefficients ofeach frame and the information of the motion compensation of each frame,from the compressed moving image data 301 (in the step 402).

Next, the extracting unit 304 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of the compressed movingimage from the DCT coefficients of each frame, which is included in theinformation 303 extracted by extracting unit 302, or the extracting unit901 and the DCT operation unit 902, mean value of the values, totalvalue of the values and/or difference value of the values. On the otherhand, the extracting unit 304 calculates values such as a motion vector,a mean motion vector between frames before and behind, a total motionvector, a difference vector and/or a motion vector on a whole frame fromthe information of the motion compensation. And the extracting unit 304arranges at least one value from the above mentioned values on the timeaxis and then picks out all values arranged on the time axis, or somevalues from the values arranged on the time axis at uniform intervals orirregular intervals. And then, the extracting unit 304 extracts thepicked out values as the information 305 of the unique identification ofthe compressed moving image to be used for identifying the compressedmoving image (in the step 403).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 305of the unique identification of the compressed moving image extracted bythe extracting unit 304 (in the step 404).

On the other hand, the information 308 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 404.

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 308 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 308 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

The apparatus and the method for retrieving the moving image accordingto the invention are explained the above. The M×N pixels unit and theM′×N′ pixels unit may be made to be a value equal to the pixel unit ofthe standard compression encoding system such as MPEG.

As mentioned above, according to the fifth embodiment of the invention,by executing the preprocessing (the steps 1003 and 1004) on theuncompressed moving image data 101, the information 305 of the uniqueidentification of the compressed moving image for retrieving movingimage depends on the content of the moving image 101 and the compressedmoving image 301 can be extracted by the equal processing (the step403). And identical moving image can be retrieved and identified fromthe moving image by using the information 305 of the uniqueidentification of the compressed moving image at the higher speed.

An apparatus and a method for retrieving moving image in the sixthpreferred embodiment according to the invention will be explained inFIGS. 11, 12A and 12B.

FIG. 11 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 11 withFIG. 5, FIG. 7 and FIG. 9. In FIG. 11, the apparatus for retrievingmoving image comprises an extracting unit 901 for extracting informationof motion compensation in the M x N (M and N are natural numbers) pixelsunit of moving image data 101, a DCT (Discrete Cosine Transform)operation unit 902 for obtaining a DCT coefficient by executing a DCToperation in the M′×N′ (M′ and N′ are natural numbers) pixels unit ofthe moving image data 101, a detecting unit 501 for detecting a cutpoint of the moving image data 101 as information 502 of a cut point, anextracting unit 302 for extracting information 303 of the featurequantity of compressed moving image, which includes DCT coefficients ofeach frame and information of motion compensation of each frame, fromcompressed moving image data 301 provided by the standard compressionencoding process for moving image such as MPEG, a detecting unit 701 fordetecting a cut point of the compressed moving image data 301 asinformation 702 of a cut point, an extracting unit 703 for arranging atleast one information of feature quantity from the DCT coefficients ofeach frame and the information of the motion compensation of each frame,which are included in the information 303 extracted by the extractingunit 901 and the DCT operation unit 902, or the extracting unit 302, onthe time axis and for extracting information 705 of the uniqueidentification of compressed moving image to identify the compressedmoving image from the arrangement of the information of the featurequantity on the time axis in each shot unit of the compressed movingimage data 301 divided by using the information 702 of the cut point, astoring unit 306 for storing the information 705 of the uniqueidentification of the compressed moving image extracted by theextracting unit 703, a retrieving unit 309 for retrieving identicalinformation with the information 705 of the unique identification of thecompressed moving image stored in the storing unit 306 from information708 of the unique identification of compressed moving image beforehandextracted from other moving image data, a storing unit 110 for storingthe retrieving result provided by the retrieving unit 309, and an outputunit 111 for displaying and/or printing the retrieving result stored inthe storing unit 110. On the apparatus for retrieving the moving imageaccording to the present invention constituted like the above, anoperation will be explained in the following.

FIGS. 12A and 12B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 11. Still, an identical step number is appended to the stephaving the identical process in FIGS. 12A and 12B with FIGS. 6A, 6B, 8A,8B, 10A and 10B. In FIGS. 11, 12A and 12B, at first whether inputtedmoving image data is compressed is judged (in the step 1001).

In case the inputted moving image data is the uncompressed moving imagedata 101, the extracting unit 901 for extracting the information of themotion compensation, the DCT operation unit 902 and the detecting unit501 for detecting the cut point fetch the moving image data 101 to be aretrieval key for identifying the moving image (in the step 1201).

The extracting unit 901 divides each frame of the moving image data 101into the M×N (M and N are natural numbers) pixels units, then calculatesthe information of the motion compensation in the M×N pixels unit (inthe step 1003). On the other hand, the DCT operation unit 902 divideseach frame of the moving image data 101 into the M′×N′ (M′ and N′ arenatural numbers) pixels units, then calculates the DCT coefficient byexecuting the DCT operation in the M′×N′ pixels unit (in the step 1004).

And, the detecting unit 501 detects the cut point of the moving imagedata 101 as the information 502 of the cut point (in the step 602).

The information of the motion compensation in the M×N pixels unitprovided by the extracting unit 901 and the DCT coefficient in the M′×N′pixels unit provided by the DCT operation unit 902 are sent to theextracting unit 703 for extracting the information of the uniqueidentification of the compressed moving image with the information 502of the cut point detected by the detecting unit 501 as the information303 of the feature quantity of the compressed moving image.

On the other hand, in case the inputted moving image data is thecompressed moving image data 301, the extracting unit 302 for extractingthe information of the feature quantity of the compressed moving imageand the detecting unit 701 for detecting the cut point fetch thecompressed moving image data 301 to be a retrieval key for identifyingthe compressed moving image (in the step 801). Next, the extracting unit302 extracts the information 303 of the feature quantity of thecompressed moving image, which includes the DCT coefficients of eachframe and the information of the motion compensation of each frame, fromthe compressed moving image data 301 (in the step, 402). And, thedetecting unit 701 detects the cut point of the compressed moving imagedata 301 as the information 702 of the cut point (in the step 802).

Next, the extracting unit 703 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of the compressed movingimage from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the valuesand/or difference value of the values. On the other hand, the extractingunit 703 calculates values such as a motion vector, a mean motion vectorbetween frames before and behind, a total motion vector, a differencevector and/or a motion vector on a whole frame from the information ofthe motion compensation. And the extracting unit 703 arranges at leastone value from the above mentioned values on the time axis and thenpicks out all values arranged on the time axis, or some values from thevalues arranged on the time axis at uniform intervals or irregularintervals. And then, the extracting unit 703 extracts the picked outvalues in each shot unit of the moving image data 101 divided by usingthe information 502 or in each shot unit of the compressed moving imagedata 301 divided by using the information 702 as the information 705 ofthe unique identification of the compressed moving image to be used foridentifying the compressed moving image (in the step 803).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 705of the unique identification of the compressed moving image extracted bythe extracting unit 703 (in the step 804).

On the other hand, the information 708 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 804.

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 708 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 708 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

The apparatus and the method for retrieving the moving image accordingto the invention are explained the above. The M×N pixels unit and theM′×N′ pixels unit may be made to be a value equal to the pixel unit ofthe standard compression encoding system such as MPEG.

As mentioned above, according to the sixth embodiment of the invention,by executing the preprocessing (the steps 1003 and 1004) on theuncompressed moving image data 101, the information 705 of the uniqueidentification of the compressed moving image for retrieving movingimage depends on the content of the moving image 101 and the compressedmoving image 301 can be extracted by the equal processing (the step803). And, the information 705 of the unique identification of thecompressed moving image for retrieving moving image can be extracted bythe less information quantity with utilizing the time concept of theshot according to the information 502 and the information 702. Identicalmoving image can be retrieved and identified from the moving image byusing the information 705 of the unique identification of the compressedmoving image at the higher speed.

An apparatus and a method for retrieving moving image in the seventhpreferred embodiment according to the invention will be explained inFIGS. 13 to 15B.

FIG. 13 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 13 withFIG. 3 and FIG. 9. In FIG. 13, the apparatus for retrieving moving imagecomprises an extracting unit 901 for extracting information of motioncompensation in the M×N (M and N are natural numbers) pixels unit ofmoving image data 101, an extracting unit 1301 for calculating anaverage pixel value, such as information of average color (R, G, B) orinformation of average luminance and average color difference (Y, Cb,Cr), in the M′×N′ (M′ and N′ are natural numbers) pixels unit, anextracting unit 1302 for extracting information 1303 of the featurequantity of compressed moving image, which includes DC (Direct Current)components of DCT (Discrete Cosine Transform) coefficients of each frameand information of motion compensation of each frame, from compressedmoving image data 301 provided by the standard compression encodingprocess for moving image such as MPEG, an extracting unit 1304 forarranging at least one information of feature quantity from the DCcomponents of the DCT coefficients of each frame and the information ofthe motion compensation of each frame or the average pixel value, whichare included in the information 1303 extracted by the extracting unit901 and the extracting unit 1301, or the extracting unit 1302, on thetime axis and for extracting information 305 of the uniqueidentification of compressed moving image to identify the compressedmoving image from the arrangement of the information of the featurequantity on the time axis, a storing unit 306 for storing theinformation 305 of the unique identification of the compressed movingimage extracted by the extracting unit 1304, a retrieving unit 309 forretrieving identical information with the information 305 of the uniqueidentification of the compressed moving image stored in the storing unit306 from information 308 of the unique identification of compressedmoving image beforehand extracted from other moving image data, astoring unit 110 for storing the retrieving result provided by theretrieving unit 309, and an output unit 111 for displaying and/orprinting the retrieving result stored in the storing unit 110.

FIG. 14 is a conceptual scheme showing the relationship between the DCT(Discrete Cosine Transform) operation and the IDCT (Inverse DiscreteCosine Transform) operation, which are used in the internationalstandard encoding system such as MPEG, and frame image.

The expression of the DCT operation is shown in the followingEXPRESSION 1. $\begin{matrix}{{F\left( {u,v} \right)} = {\left\{ {2{C(u)}{{C(v)}/N}} \right\}{\sum\limits_{x = 0}^{N - 1}{\sum\limits_{y = 0}^{N - 1}{\cos\left\{ {\left( {{2x} + 1} \right)u\quad{\pi/2}N} \right\}\cos\left\{ {\left( {{2y} + 1} \right)v\quad{\pi/2}N} \right\}}}}}} & \left\lbrack {{EXPRESSION}\quad 1} \right\rbrack\end{matrix}$

And, the expression of the IDCT operation is shown in the followingEXPRESSION 2. $\begin{matrix}{{f\left( {x,y} \right)} = {\left\{ {2/N} \right\}{\sum\limits_{u = 0}^{N - 1}{\sum\limits_{v = 0}^{N - 1}{{C(u)}{C(v)}\cos\left\{ {\left( {{2x} + 1} \right)u\quad{\pi/2}N} \right\}\cos\left\{ {\left( {{2y} + 1} \right)v\quad{\pi/2}N} \right\}}}}}} & \left\lbrack {{EXPRESSION}\quad 2} \right\rbrack\end{matrix}$

However, in the above mentioned EXPRESSIONs 1 and 2,

“N” is showing a block size of intra-code;

“x” and “y” are showing a coordinate in a block;

“f (x,y)” is showing a pixel value of the coordinate (x, y) in a block;

“u” and “v” are showing a coordinate after the DCT conversion;

“F (u, v)” is showing a DCT coefficient value of the coordinate (u, v)after the DCT conversion;

“C (u)” is made 1/√ 2, if u=0;

-   -   1, if u≠0; and

“C (v)” is made 1/√ 2, if v=0;

-   -   1, if v≠0.

In FIG. 14, for example, in case the block size N of one frame is 8 andthe coordinate (u, v) after the DCT conversion is (0, 0), F (u, v) inthe EXPRESSION 1 is shown in the following EXPRESSION 3. $\begin{matrix}{{F\left( {0,0} \right)} = {\left( {1/8} \right){\sum\limits_{x = 0}^{7}{\sum\limits_{y = 0}^{7}{f\left( {x,y} \right)}}}}} & \left\lbrack {{EXPRESSION}\quad 3} \right\rbrack\end{matrix}$

And more, an average pixel value in the block is shown in the followingEXPRESSION 4. $\begin{matrix}{\begin{matrix}{{Average}\quad{Pixel}} \\{{Value}\quad{in}\quad{the}\quad{block}}\end{matrix} = {{\sum\limits_{x = 0}^{7}{\sum\limits_{y = 0}^{7}{{{f\left( {x,y} \right)}/8} \times 8}}}\quad = {{F\left( {0,0} \right)}/8}}} & \left\lbrack {{EXPRESSION}\quad 4} \right\rbrack\end{matrix}$

As shown in the above mentioned EXPRESSION 3 and 4, the DC component F(u, v) of the DCT coefficient becomes a value provided by multiplyingthe average pixel value in the block by the quantization number (N) ofthe block. That is to say, the DC components F (u, v) of the DCTcoefficients extracted by the extracting unit 1302 for extracting theinformation of the feature quantity of the compressed moving image andthe average pixel value provided by the extracting unit 1301 forextracting the average value of the block unit may be handled asidentical information data considering the quantization number of theblock. Specifically, the information 1303 extracted by the extractingunit 1302, and the information 1303 depends on the information of themotion compensation extracted by the extracting unit 901 and the averagepixel value extracted by the extracting unit 1301 may be consideredinformation of data having an identical content composition.

On the apparatus for retrieving the moving image according to thepresent invention constituted like the above, an operation will beexplained in the following.

FIGS. 15A and 15B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 13. Still, an identical step number is appended to the stephaving the identical process in FIGS. 15A and 15B with FIGS. 4A, 4B, 10Aand 10B. In FIGS. 13, 15A and 15B, at first whether inputted movingimage data is compressed is judged. (in the step 1001).

In case the inputted moving image data is the uncompressed moving imagedata 101, the extracting unit 901 for extracting the information of themotion compensation and the extracting unit 1301 for extracting theaverage value of the block unit fetch the moving image data 101 to be aretrieval key for identifying the moving image (in the step 1501).

The extracting unit 901 divides each frame of the moving image data 101into the M×N (M and N are natural numbers) pixels units, then calculatesthe information of the motion compensation in the M×N pixels unit (inthe step 1003). On the other hand, the extracting unit 1301 divides eachframe of the moving image data 101 into the M′×N′ (M′ and N′ are naturalnumbers) pixels units, then calculates the average pixel value in theM′×N′ pixels unit (in the step 1502).

The information of the motion compensation in the M×N pixels unitprovided by the extracting unit 901 and the average pixel value in theM′×N′ pixels unit provided by the extracting unit 1301 are sent to theextracting unit 1304 for extracting the information of the uniqueidentification of the compressed moving image as the information 1303 ofthe feature quantity of the compressed moving image.

On the other hand, in case the inputted moving image data is thecompressed moving image data 301, the extracting unit 1302 forextracting the information of the feature quantity of the compressedmoving image fetches the compressed moving image data 301 to be aretrieval key for identifying the compressed moving image (in the step401). And the extracting unit 1302 extracts the information 1303 of thefeature quantity of the compressed moving image, which includes the DCcomponents of the DCT coefficients of each frame and the information ofthe motion compensation of each frame, from the compressed moving imagedata 301 (in the step 1503).

Next, the extracting unit 1304 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of compressed movingimage from the DC components of the DCT coefficients of each frameand/or values of the average pixel value of each frame, which areincluded in the information 1303, mean value of the values, total valueof the values and/or difference value of the values. On the other hand,the extracting unit 1304 calculates values such as a motion vector, amean motion vector between frames before and behind, a total motionvector, a difference vector and/or a motion vector on a whole frame fromthe information of the motion compensation. And the extracting unit 1304arranges at least one value from the above mentioned values on the timeaxis and then picks out all values arranged on the time axis, or somevalues from the values arranged on the time axis at uniform intervals orirregular intervals. And then, the extracting unit 1304 extracts thepicked out values as the information 305 of the unique identification ofthe compressed moving image to be used for identifying the compressedmoving image (in the step 1504).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 305of the unique identification of the compressed moving image extracted bythe extracting unit 1304 (in the step 404).

On the other hand, the information 308 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 404.

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 308 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 308 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

The apparatus and the method for retrieving the moving image accordingto the invention are explained the above. The above mentioned extractingunit 1304 calculates values of physical information of the featurequantity of the compressed moving image from the DC components of theDCT coefficients of each frame and/or values of the average pixel valueof each frame, which are included in the information 1303, mean value ofthe values, total value of the values and/or difference value of thevalues. And the extracting unit 1304 calculates values such as a motionvector, a mean motion vector between frames before and behind, a totalmotion vector, a difference vector and/or a motion vector on a wholeframe from the information of the motion compensation. However, theextracting unit 1304 may calculate the above mentioned values by using apart or multiple parts of the physical information of the featurequantity of the compressed moving image from the DC components of theDCT coefficients of each frame and/or the average pixel value of eachframe, which are included in the information 1303, and the abovementioned vectors by using a part or multiple parts of the informationof the motion compensation.

And more, the extracting unit 1304 may execute the weighting on thephysical information of the feature quantity of the compressed movingimage from the DC components of the DCT coefficients of each frameand/or on the average pixel value of each frame, then may calculate theabove mentioned values from the weighted information of the featurequantity of the compressed moving image and/or the weighted averagepixel value. On the other hand, the extracting unit 1304 may execute theweighting on the information of the motion compensation and maycalculate the above mentioned vectors from the weighted information ofthe motion compensation.

Moreover, in the extracting unit 1304, values provided by quantizing theabove mentioned values of the physical information or of the averagepixel value or the like, and values provided by quantizing the abovementioned motion vectors and the like may be applied.

Further more, the M×N pixels unit and the M′×N′ pixels unit may be madeto be a value equal to the pixel unit of the standard compressionencoding system such as MPEG.

As mentioned above, according to the seventh embodiment of theinvention, by executing the preprocessing on the moving image, theinformation 305 of the unique identification of the compressed movingimage for retrieving the moving image depends on the content of themoving image data 101 and the compressed moving image data 301 can beextracted by the less information quantity and by executing the equalprocess for the moving image data 101 and the compressed moving imagedata 301. And, identical moving image can be retrieved and identifiedfrom the moving image by using the information 305 of the uniqueidentification of the compressed moving image at the higher speed.

An apparatus and a method for retrieving moving image in the eighthpreferred embodiment according to the invention will be explained inFIGS. 16, 17A and 17B.

FIG. 16 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 16 withFIG. 5, FIG. 7 and FIG. 13. In FIG. 16, the apparatus for retrievingmoving image comprises an extracting unit 901 for extracting informationof motion compensation in the M x N (M and N are natural numbers) pixelsunit of moving image data 101, an extracting unit 1301 for calculatingan average pixel value, such as information of average color (R, G, B)or information of average luminance and average color difference (Y, Cb,Cr), in the M′×N′ (M′ and N′ are natural numbers) pixels unit, adetecting unit 501 for detecting a cut point of the moving image data101 as information 502 of a cut point, an extracting unit 1302 forextracting information 1303 of the feature quantity of compressed movingimage, which includes DC (Direct Current) components of DCT (DiscreteCosine Transform) coefficients of each frame and information of motioncompensation of each frame, from compressed moving image data 301provided by the standard compression encoding process for moving imagesuch as MPEG, a detecting unit 701 for detecting a cut point of thecompressed moving image data 301 as information 702 of a cut point, anextracting unit 1603 for arranging at least one information of featurequantity from the DC components of the DCT coefficients of each frameand the information of the motion compensation of each frame or theaverage pixel value, which are included in the information 1303extracted by the extracting unit 901 and the extracting unit 1301, orthe extracting unit 1302, on the time axis and for extractinginformation 705 of the unique identification of compressed moving imageto identify the compressed moving image from the arrangement of theinformation of the feature quantity on the time axis in each shot unitof the moving image data 101 divided by using the information 502 of thecut point or in each shot unit of the compressed moving image data 301divided by using the information 702 of the cut point, a storing unit306 for storing the information 705 of the unique identification of thecompressed moving image extracted by the extracting unit 1603, aretrieving unit 309 for retrieving identical information with theinformation 705 of the unique identification of the compressed movingimage stored in the storing unit 306 from information 708 of the uniqueidentification of compressed moving image beforehand extracted fromother moving image data, a storing unit 110 for storing the retrievingresult provided by the retrieving unit 309, and an output unit 111 fordisplaying and/or printing the retrieving result stored in the storingunit 110. On the apparatus for retrieving the moving image according tothe present invention constituted like the above, an operation will beexplained in the following.

FIGS. 17A and 17B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 16. Still, an identical step number is appended to the stephaving the identical process in FIGS. 17A and 17B with FIGS. 6A, 6B, 8A,8B, 15A and 15B. In FIGS. 16, 17A and 17B, at first whether inputtedmoving image data is compressed is judged (in the step 1001).

In case the inputted moving image data is the uncompressed moving imagedata 101, the extracting unit 901 for extracting the information of themotion compensation, the extracting unit 1301 for extracting the averagevalue of the block unit and the detecting unit 501 for detecting the cutpoint fetch the moving image data 101 to be a retrieval key foridentifying the moving image (in the step 1701).

The extracting unit 901 divides each frame of the moving image data 101into the M×N (M and N are natural numbers) pixels units, then calculatesthe information of the motion compensation in the M×N pixels unit (inthe step 1003). And, the extracting unit 1301 divides each frame of themoving image data 101 into the M′×N′ (M′ and N′ are natural numbers)pixels units, then calculates the average pixel value in the M′×N′pixels unit (in the step 1502). And more, the detecting unit 501 detectsthe cut point of the moving image data 101 as the information 502 of thecut point (in the step 602).

The information of the motion compensation in the M×N pixels unitprovided by the extracting unit 901 and the average pixel value in theM′×N′ pixels unit provided by the extracting unit 1301 are sent to theextracting unit 1603 for extracting the information of the uniqueidentification of the compressed moving image with the information 502of the cut point detected by the detecting unit 501 as the information1303 of the feature quantity of the compressed moving image.

On the other hand, in case the inputted moving image data is thecompressed moving image data 301, the extracting unit 1302 forextracting the information of the feature quantity of the compressedmoving image and the detecting unit 701 for detecting the cut pointfetch the compressed moving image data 301 to be a retrieval key foridentifying the compressed moving image (in the step 801). And theextracting unit 1302 extracts the information 1303 of the featurequantity of the compressed moving image, which includes the DCcomponents of the DCT coefficients of each frame and the information ofthe motion compensation of each frame, from the compressed moving imagedata 301 (in the step 1503). And then, the detecting unit 701 detectsthe cut point of the compressed moving image data 301 as the informationof the cut point 702 (in the step 802).

Next, the extracting unit 1603 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of the compressed movingimage from the DC components of the DCT coefficients of each frameand/or values of the average pixel value of each frame, which areincluded in the information 1303, mean value of the values, total valueof the values and/or difference value of the values. On the other hand,the extracting unit 1603 calculates values such as a motion vector, amean motion vector between frames before and behind, a total motionvector, a difference vector and/or a motion vector on a whole frame fromthe information of the motion compensation. And the extracting unit 1603arranges at least one value from the above mentioned values on the timeaxis and then picks out all values arranged on the time axis, or somevalues from the values arranged on the time axis at uniform intervals orirregular intervals. And then, the extracting unit 1603 extracts thepicked out values in each shot unit of the moving image data 101 dividedby using the information 502 of the cut point or in each shot unit ofthe compressed moving image data 301 divided by using the information702 of the cut point as the information 705 of the unique identificationof the compressed moving image to be used for identifying the compressedmoving image (in the step 1702).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 705of the unique identification of the compressed moving image extracted bythe extracting unit 1603 (in the step 804).

On the other hand, the information 708 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 804.

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 708 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 708 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

The apparatus and the method for retrieving the moving image accordingto the invention are explained the above. The M×N pixels unit and theM′×N′ pixels unit may be made to be a value equal to the pixel unit ofthe standard compression encoding system such as MPEG.

As mentioned above, according to the eighth embodiment of the invention,by utilizing the time concept of the shot according to the informationof the cut point and by executing the preprocessing on the moving image,information 1303 of the unique identification of the compressed movingimage for retrieving moving image depends on the content of the movingimage data 101 and the compressed moving image data 301 can be extractedby the less information quantity and by executing the equal process forthe moving image data 101 and the compressed moving image data 301. And,identical moving image can be retrieved and identified from the movingimage by using the information 1303 of the unique identification of thecompressed moving image at the higher speed.

An apparatus and a method for retrieving moving image in the ninthpreferred embodiment according to the invention will be explained inFIGS. 18, 19A and 19B.

FIG. 18 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 18 withFIG. 3 and FIG. 9. In FIG. 18, the apparatus for retrieving moving imagecomprises a DCT (Discrete Cosine Transform) operation unit 902 forobtaining a DCT coefficient by executing a DCT operation in the M×N (Mand N are natural numbers) pixels unit of moving image data 101 asinformation 303 of the feature quantity of compressed moving image, anextracting unit 302 for extracting information 303 of the featurequantity of compressed moving image, which includes the DCT coefficientsof each frame, from compressed moving image data 301 provided by thestandard compression encoding process for moving image such as MPEG, anextracting unit 304 for arranging at least one information of featurequantity from the DCT coefficients of each frame, which is included inthe information 303 extracted by the DCT operation unit 902 or by theextracting unit 302, on the time axis and for extracting information 305of the unique identification of compressed moving image to identify thecompressed moving image from the arrangement of the information offeature quantity on the time axis, a storing unit 306 for storing theinformation 305 of the unique identification of the compressed movingimage extracted by the extracting unit 304, a retrieving unit 309 forretrieving identical information with the information 305 of the uniqueidentification of the compressed moving image stored in the storing unit306 from information 308 of the unique identification of compressedmoving image beforehand extracted from other moving image data, astoring unit 110 for storing the retrieving result provided by theretrieving unit 309, and an output unit 111 for displaying and/orprinting the retrieving result stored in the storing unit 110. On theapparatus for retrieving the moving image according to the presentinvention constituted like the above, an operation will be explained inthe following.

FIGS. 19A and 19B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 18. Still, an identical step number is appended to the stephaving the identical process in FIGS. 19A and 19B with FIGS. 10A and10B. In FIGS. 18, 19A and 19B, at first whether inputted moving imagedata is compressed is judged. (in the step 1001).

In case the inputted moving image data is the uncompressed moving imagedata 101, the DCT operation unit 902 fetches the moving image data 101to be a retrieval key for identifying the moving image (in the step1901).

The DCT operation unit 902 divides each frame of the moving image data101 into the M×N (M and N are natural numbers) pixels units, thencalculates the DCT coefficient by executing the DCT operation in the M×Npixels unit as the information 303 of the feature quantity of thecompressed moving image (in the step 1902).

On the other hand, in case the inputted moving image data is thecompressed moving image data 301, the extracting unit 302 for extractingthe information of the feature quantity of the compressed moving imagefetches the compressed moving image data 301 to be a retrieval key foridentifying the compressed moving image (in the step 401). And theextracting unit 302 extracts the information 303 of the feature quantityof the compressed moving image, which includes the DCT coefficients ofeach frame, from the compressed moving image data 301 as (in the step1903).

Next, the extracting unit 304 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of the compressed movingimage from the DCT coefficients of each frame, which is included in theinformation 303 extracted by the extracting unit 902, mean value of thevalues, total value of the values and/or difference value of the values.And the extracting unit 304 arranges at least one value from the abovementioned values on the time axis and then picks out all values arrangedon the time axis, or some values from the values arranged on the timeaxis at uniform intervals or irregular intervals. And then, theextracting unit 304 extracts the picked out values as the information305 of the unique identification of the compressed moving image to beused for identifying the compressed moving image (in the step 1904).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 305of the unique identification of the compressed moving image extracted bythe extracting unit 304 (in the step 404).

On the other hand, the information 308 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 404.

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 308 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 308 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

The apparatus and the method for retrieving the moving image accordingto the invention are explained the above. The M×N pixels unit may bemade to be a value equal to the pixel unit of the standard compressionencoding system such as MPEG.

As mentioned above, according to the ninth embodiment of the invention,by executing the preprocessing on the moving image 101, the information305 of the unique identification of the compressed moving image forretrieving moving image depends on the content of the moving image data101 and the compressed moving image data 301 can be extracted byexecuting the equal process. And, identical moving image can beretrieved and identified from the moving image by using the information305 of the unique identification of the compressed moving image at thehigher speed.

An apparatus and a method for retrieving moving image in the tenthpreferred embodiment according to the invention will be explained inFIGS. 20, 21A and 21B.

FIG. 20 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 20 withFIG. 11. In FIG. 20, the apparatus for retrieving moving image comprisesa DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCTcoefficient as information 303 of the feature quantity of compressedmoving image by executing a DCT operation in the M×N (M and N arenatural numbers) pixels unit of moving image data 101, a detecting unit501 for detecting a cut point of the moving image data 101 asinformation 502 of a cut point, an extracting unit 302 for extractinginformation 303 of the feature quantity of compressed moving image,which includes DCT coefficients of each frame, from compressed movingimage data 301 provided by the standard compression encoding process formoving image such as MPEG, a detecting unit 701 for detecting a cutpoint of the compressed moving image data 301 as information 702 of acut point, an extracting unit 703 for arranging at least one informationof feature quantity from the DCT coefficients of each frame, which isincluded in the information 303 extracted by the DCT operation unit 902or the extracting unit 302, on the time axis and for extractinginformation 705 of the unique identification of compressed moving imageto identify the compressed moving image from the arrangement of theinformation of the feature quantity on the time axis in each shot unitof the compressed moving image data 301 divided by using the information502 of the cut point and/or the information 702 of the cut point, astoring unit 306 for storing the information 705 of the uniqueidentification of the compressed moving image extracted by theextracting unit 703, a retrieving unit 309 for retrieving identicalinformation with the information 705 of the unique identification of thecompressed moving image stored in the storing unit 306 from information708 of the unique identification of compressed moving image beforehandextracted from other moving image data, a storing unit 110 for storingthe retrieving result provided by the retrieving unit 309, and an outputunit 111 for displaying and/or printing the retrieving result stored inthe storing unit 110. On the apparatus for retrieving the moving imageaccording to the present invention constituted like the above, anoperation will be explained in the following.

FIGS. 21A and 21B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 20. Still, an identical step number is appended to the stephaving the identical process in FIGS. 21A and 21B with FIGS. 12A, 12B,19A and 19B. In FIGS. 20, 21A and 21B, at first whether inputted movingimage data is compressed is judged. (in the step 1001).

In case the inputted moving image data is the uncompressed moving imagedata 101, the DCT operation unit 902 and the detecting unit 501 fordetecting the cut point fetch the moving image data 101 to be aretrieval key for identifying the moving image (in the step 2101).

The DCT operation unit 902 divides each frame of the moving image data101 into the M×N (M and N are natural numbers) pixels units, thencalculates the DCT coefficient by executing the DCT operation in the M×Npixels unit as the information 303 of the feature quantity of thecompressed moving image (in the step 1902). On the other hand, thedetecting unit 501 for detecting the cut point detects the cut point ofthe moving image data 101 as the information 502 of the cut point (inthe step 602).

The DCT coefficient in the M×N pixels unit provides by the DCT operationunit 902 is sent to the extracting unit 703 for extracting theinformation of the unique identification of the compressed moving imagewith the information 502 detected by the detecting unit 501 as theinformation 303 of the feature quantity of the compressed moving image.

On the other hand, in case the inputted moving image data is thecompressed moving image data 301, the extracting unit 302 for extractingthe information of the feature quantity of the compressed moving imageand the detecting unit 701 for detecting the cut point fetch thecompressed moving image data 301 to be a retrieval key for identifyingthe compressed moving image (in the step 801). Next, the extracting unit302 extracts the information 303 of the feature quantity of thecompressed moving image, which includes the DCT coefficients of eachframe, from the compressed moving image data 301 (in the step 1903).And, the detecting unit 701 detects the cut point of the compressedmoving image data 301 as the information 702 of the cut point (in thestep 802).

Next, the extracting unit 703 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of the compressed movingimage from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the valuesand/or difference value of the values. And the extracting unit 703arranges at least one value from the above mentioned values on the timeaxis and then picks out all values arranged on the time axis, or somevalues from the values arranged on the time axis at uniform intervals orirregular intervals. And then, the extracting unit 703 extracts thepicked out values in each shot unit of the moving image data 101 dividedby using the information 502 or in each shot unit of the compressedmoving image data 301 divided by using the information 702 as theinformation 705 of the unique identification of the compressed movingimage to be used for identifying the compressed moving image (in thestep 2102).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 705of the unique identification of the compressed moving image extracted bythe extracting unit 703 (in the step 804).

On the other hand, the information 708 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 804.

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 708 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 708 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

The apparatus and the method for retrieving the moving image accordingto the invention are explained the above. The M×N pixels unit may bemade to be a value equal to the pixel unit of the standard compressionencoding system such as MPEG.

As mentioned above, according to the tenth embodiment of the invention,by executing the preprocessing on the uncompressed moving image data101, the information 705 of the unique identification of the compressedmoving image for retrieving moving image depends on the content of themoving image 101 and the compressed moving image 301 can be extracted bythe equal processing. And, the information 705 for retrieving movingimage can be extracted by the less information quantity with utilizingthe time concept of the shot according to the information 502 and theinformation 702. Identical moving image can be retrieved and identifiedfrom the moving image by using the information 705 of the uniqueidentification of the compressed moving image at the higher speed.

An apparatus and a method for retrieving moving image in the eleventhpreferred embodiment according to the invention will be explained inFIGS. 22, 23A and 23B.

FIG. 22 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 22 withFIG. 13. In FIG. 22, the apparatus for retrieving moving image comprisesan extracting unit 1301 for calculating an average pixel value, such asinformation of average color (R, G, B) or information of averageluminance and average color difference (Y, Cb, Cr), in the M×N (M and Nare natural numbers) pixels unit, an extracting unit 1302 for extractinginformation 1303 of the feature quantity of compressed moving image,which includes DC (Direct Current) components of DCT (Discrete CosineTransform) coefficients of each frame, from compressed moving image data301 provided by the standard compression encoding process for movingimage such as MPEG, an extracting unit 1304 for arranging at least oneinformation of feature quantity from the DC components of the DCTcoefficients of each frame or the average pixel value, which is includedin the information 1303 extracted by the extracting unit 1301 and/or theextracting unit 1302, on the time axis and for extracting information305 of the unique identification of compressed moving image to identifycompressed moving image from the arrangement of the information of thefeature quantity on the time axis, a storing unit 306 for storing theinformation 305 of the unique identification of the compressed movingimage extracted by the extracting unit 1304, a retrieving unit 309 forretrieving identical information with the information 305 of the uniqueidentification of the compressed moving image stored in the storing unit306 from information 308 of the unique identification of compressedmoving image beforehand extracted from other moving image data, astoring unit 110 for storing the retrieving result provided by theretrieving unit 309, and an output unit 111 for displaying and/orprinting the retrieving result stored in the storing unit 110. On theapparatus for retrieving the moving image according to the presentinvention constituted like the above, an operation will be explained inthe following.

FIGS. 23A and 23B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 22. Still, an identical step number is appended to the stephaving the identical process in FIGS. 23A and 23B with FIGS. 15A and15B. In FIGS. 22, 23A and 23B, at first whether inputted moving imagedata is compressed is judged. (in the step 1001).

In case the inputted moving image data is the uncompressed moving imagedata 101, the extracting unit 1301 for extracting the average value ofthe block unit fetches the moving image data 101 to be a retrieval keyfor identifying the moving image (in the step 2301).

The extracting unit 1301 divides each frame of the moving image data 101into the M×N (M and N are natural numbers) pixels units, then calculatesthe average pixel value in the M×N pixels unit as the information 1303of the feature quantity of the compressed moving image (in the step1502).

On the other hand, in case the inputted moving image data is thecompressed moving image data 301, the extracting unit 1302 forextracting the information of the feature quantity of the compressedmoving image fetches the compressed moving image data 301 to be aretrieval key for identifying the compressed moving image (in the step401). Then, the extracting unit 1302 extracts the information 1303 ofthe feature quantity of the compressed moving image, which includes theDC components of the DCT coefficients of each frame, from the compressedmoving image data 301 (in the step 1503).

Next, the extracting unit 1304 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of compressed movingimage from the DC components of the DCT coefficients of each frameand/or values of the average pixel value of each frame, which areincluded in the information 1303, mean value of the values, total valueof the values and i or difference value of the values. And theextracting unit 1304 arranges at least one value from the abovementioned values on the time axis and then picks out all values arrangedon the time axis, or some values from the values arranged on the timeaxis at uniform intervals or irregular intervals. And then, theextracting unit 1304 extracts the picked out values as the information305 of the unique identification of the compressed moving image to beused for identifying the compressed moving image (in the step 1504).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 305of the unique identification of the compressed moving image extracted bythe extracting unit 1304 (in the step 404).

On the other hand, the information 308 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 404.

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 308 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orsprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 308 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

The apparatus and the method for retrieving the moving image accordingto the invention are explained the above. The above mentioned extractingunit 1304 calculates values of the physical information of the featurequantity of the compressed moving image from the DC components of theDCT coefficients of each frame or values of the average pixel value ofeach frame, which are included in the information 1303, mean value ofthe values, total value of the values and/ or difference value of thevalues. However, the extracting unit 1304 may calculate the abovementioned values by using a part or multiple parts of the physicalinformation of the feature quantity of the compressed moving image fromthe DC components of the DCT coefficients of each frame or the averagepixel value of each frame, which are included in the information 1303.

And more, the extracting unit 1304 may execute the weighting on thephysical information of the feature quantity of the compressed movingimage from the DC components of the DCT coefficients of each frame orthe average pixel value of each frame, which are included in theinformation 1303, then may calculate the above mentioned values by theweighted information of the feature quantity of the compressed movingimage or the weighted average pixel value.

Moreover, in the extracting unit 1304, values provided by quantizing theabove mentioned physical information or average pixel value or the likemay be applied.

Further more, the M×N pixels unit may be made to be a value equal to thepixel unit of the standard compression encoding system such as MPEG.

As mentioned above, according to the eleventh embodiment of theinvention, by executing the preprocessing on the moving image, theinformation 305 of the unique identification of the compressed movingimage for retrieving the moving image depends on the content of themoving image data 101 and the compressed moving image data 301 can beextracted by the less information quantity and by executing the equalprocess for the moving image data 101 and the compressed moving imagedata 301. And, identical moving image can be retrieved and identifiedfrom the moving image by using the information 305 of the uniqueidentification of the compressed moving image at the higher speed.

An apparatus and a method for retrieving moving image in the twelfthpreferred embodiment according to the invention will be explained inFIGS. 24, 25A and 25B.

FIG. 24 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 24 withFIG. 16. In FIG. 24, the apparatus for retrieving moving image comprisesan extracting unit 1301 for calculating an average pixel value, such asinformation of average color (R, G, B) or information of averageluminance and average color difference (Y, Cb, Cr), in the M×N (M and Nare natural numbers) pixels unit of moving image data 101 as information1303 of the feature quantity of compressed moving image, a detectingunit 501 for detecting a cut point of the moving image data 101 asinformation 502 of a cut point, an extracting unit 1302 for extractinginformation 1303 of the feature quantity of compressed moving image,which includes DC (Direct Current) components of DCT (Discrete CosineTransform) coefficients of each frame, from compressed moving image data301 provided by the standard compression encoding process for movingimage such as MPEG, a detecting unit 701 for detecting a cut point ofthe compressed moving image data 301 as information 702 of a cut point,an extracting unit 1603 for arranging at least one information offeature quantity from the DC components of the DCT coefficients of eachframe or the average pixel value, which is included in the information1303 extracted by the extracting unit 1301 and/or the extracting unit1302, on the time axis and for extracting information 705 of the uniqueidentification of compressed moving image to identify the compressedmoving image from the arrangement of the information of the featurequantity on the time axis in each shot unit of the moving image data 101divided by using the information 502 of the cut point or in each shotunit of the compressed moving image data 301 divided by using theinformation 702 of the cut point, a storing unit 306 for storing theinformation 705 of the unique identification of the compressed movingimage extracted by the extracting unit 1603, a retrieving unit 309 forretrieving identical information with the information 705 of the uniqueidentification of the compressed moving image stored in the storing unit306 from information 708 of the unique identification of compressedmoving image beforehand extracted from other moving image data, astoring unit 110 for storing the retrieving result provided by theretrieving unit 309, and an output unit 111 for displaying and/orprinting the retrieving result stored in the storing unit 110. On theapparatus for retrieving the moving image according to the presentinvention constituted like the above, an operation will be explained inthe following.

FIGS. 25A and 25B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 24. Still, an identical step number is appended to the stephaving the identical process in FIGS. 25A and 25B with FIGS. 17A and17B. In FIGS. 24, 25A and 25B, at first whether inputted moving imagedata is compressed is judged (in the step 1001).

In case the inputted moving image data is the uncompressed moving imagedata 101, the extracting unit 1301 for extracting the average value ofthe block unit and the detecting unit 501 for detecting the cut pointfetch the moving image data 101 to be a retrieval key for identifyingthe moving image (in the step 2501).

The extracting unit 1301 divides each frame of the moving image data 101into the M×N (M and N are natural numbers) pixels units, then calculatesthe average pixel value in the M×N pixels unit (in the step 1502). And,the detecting unit 501 detects the cut point of the moving image data101 as the information 502 of the cut point (in the step 602).

On the other hand, in case the inputted moving image data is thecompressed moving image data 301, the extracting unit 1302 forextracting the information of the feature quantity of the compressedmoving image and the detecting unit 701 for detecting the cut pointfetch the compressed moving image data 301 to be a retrieval key foridentifying the compressed moving image (in the step 801). Then, theextracting unit 1302 extracts the information 1303.of the featurequantity of the compressed moving image, which includes the DCcomponents of the DCT coefficients of each frame, from the compressedmoving image data 301 (in the step 1503). And, the detecting unit 701detests the cut point of the compressed moving image data 301 as theinformation 702 of the cut point (in the step 802).

Next, the extracting unit 1603 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of compressed movingimage from the DC components of the DCT coefficients of each frameand/or the average pixel value of each frame, which are included in theinformation 1303, mean value of the values, total value of the valuesand/or difference value of the values. And the extracting unit 1603arranges at least one value from the above mentioned values on the timeaxis and then picks out all values arranged on the time axis, or somevalues from the values arranged on the time axis at uniform intervals orirregular intervals. And then, the extracting unit 1603 extracts thepicked out values in each shot unit of the moving image data 101 dividedby using the information 502 of the cut point or in each shot unit ofthe compressed moving image data 301 divided by using the information702 of the cut point as the information 705 of the unique identificationof the compressed moving image to be used for identifying the compressedmoving image (in the step 1702).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 705of the unique identification of the compressed moving image extracted bythe extracting unit 1603 (in the step 804).

On the other hand, the information 708 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 804.

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 708 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 708 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

The apparatus and the method for retrieving the moving image accordingto the invention are explained the above. The M×N pixels unit may bemade to be a value equal to the pixel unit of the standard compressionencoding system such as MPEG.

As mentioned above, according to the twelfth embodiment of theinvention, by utilizing the time concept of the shot according to theinformation of the cut point and executing the simple preprocessing onthe moving image, the information 1303 of the unique identification ofthe compressed moving image for retrieving the moving image depends onthe content of the moving image data 101 and the compressed moving imagedata 301 can be extracted by the less information quantity and byexecuting the equal process for the moving image data 101 and thecompressed moving image data 301. And, identical moving image can beretrieved and identified from the moving image by using the information1303 of the unique identification of the compressed moving image at thehigher speed.

An apparatus and a method for retrieving moving image in the thirteenthpreferred embodiment according to the invention will be explained inFIGS. 26, 27A and 27B.

FIG. 26 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 26 withFIG. 1 and FIG. 3. In FIG. 26, the apparatus for retrieving moving imagecomprises an extracting unit 102 for extracting information 103 of thefeature quantity of moving image, which includes physical information ofthe feature quantity of moving image such as information of luminanceand color difference (Y, Cb, Cr) of each frame and/or information ofcolor (R, G, B) of each frame, from moving image data 101, an extractingunit 104 for arranging at least one physical information included in theinformation 103 (such as the information of the luminance and the colordifference (Y, Cb, Cr) of each frame and/or the information of the color(R, G, B) of each frame) on the time axis and for extracting information105 of the unique identification of moving image to identify the movingimage from the arrangement of the physical information on the time axis,a storing unit 106 for storing the information 105 of the uniqueidentification of the moving image extracted by the extracting unit 104,a retrieving unit 109 for retrieving identical information with theinformation 105 of the unique identification of the moving image storedin the storing unit 106 from information 108 of the uniqueidentification of moving image beforehand extracted from other movingimage data, an extracting unit 302 for extracting information 303 of thefeature quantity of compressed moving image, which includes DCT(Discrete Cosine Transform) coefficients of each frame and informationof motion compensation of each frame, from compressed moving image data301 provided by the standard compression encoding process for movingimage such as MPEG, an extracting unit 304 for arranging at least oneinformation of feature quantity from the DCT coefficients of each frameand the information of the motion compensation of each frame, which areincluded in the information 303, on the time axis and for extractinginformation 305 of the unique identification of compressed moving imageto identify the compressed moving image from the arrangement of theinformation of the feature quantity on the time axis, a storing unit 306for storing the information 305 of the unique identification of thecompressed moving image extracted by the extracting unit 304, aretrieving unit 309 for retrieving identical information with theinformation 305 of the unique identification of the compressed movingimage stored in the storing unit 306 from information 308 of the uniqueidentification of compressed moving image beforehand extracted fromother moving image data, a judging unit 2602 for judging whetherinputted digital moving image data 2601 is the moving image data 101 orthe compressed moving image data 301 and for outputting the moving imagedata 101 to the extracting unit 102 and the compressed moving image data301 to the extracting unit 302, a storing unit 110 for storing aretrieving result provided by the retrieving unit 109 and/or by theretrieving unit 309, and an output unit 111 for displaying and/orprinting the retrieving result stored in the storing unit 110.

FIGS. 27A and 27B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 26. Still, an identical step number is appended to the stephaving the identical process in FIGS. 27A and 27B with FIGS. 2A, 2B, 4Aand 4B. In FIGS. 26, 27A and 27B, at first the compressed/uncompressedjudging unit 2602 acquires the digital moving image data 2601 (in thestep 2701). Then, the compressed/uncompressed judging unit 2602 judgeswhether the acquired digital moving image data 2601 is the uncompressedmoving image data 101 or the compressed moving image data 301 (in thestep 2702).

In case the digital moving image data 2601 is the uncompressed movingimage data 101, the following process (the steps 202 to 210) isexecuted.

The extracting unit 102 extracts the information 103 of the featurequantity of the moving image, which includes the physical information ofthe feature quantity of the moving image such as the information of theluminance and the color difference (Y, Cb, Cr) of each frame and/or theinformation of the color (R, G, B) of each frame, from the moving imagedata 101 (in the step 202).

Next, the extracting unit 104 for extracting the information of theunique identification of the moving image arranges at least one valuefrom values of the physical information of the feature quantity of themoving image (such as the information of the luminance and the colordifference (Y, Cb, Cr) of each frame and/or the information of the color(R, G, B) of each frame) based on the information 103, mean value of thevalues, total value of the values and/or difference value of the valueson the time axis. Then, the extracting unit 104 picks out all valuesarranged on the time axis, or some values from the values arranged onthe time axis at uniform intervals or irregular intervals. And then, theextracting unit 104 extracts the picked out values as the information105 of the unique identification of the moving image to be used foridentifying the moving image (in the step 203).

The storing unit 106 for storing the information of the uniqueidentification of the moving image stores the information 105 of theunique identification of the moving image extracted by the extractingunit 104 (in the step 204).

On the other hand, the information 108 of the unique identification ofthe moving image is extracted from other moving image data (the objectmoving image data for retrieving) (in the step 205). This extractingprocess can be executed by the processing equal to the foregoing steps201 to 204 (in FIGS. 2A and 2B).

Then, the retrieving unit 109 for retrieving the information of theunique identification of the moving image compares the information 105of the unique identification of the moving image stored in the storingunit 106 with the information 108 of the unique identification of themoving image extracted from the other moving image data (in the step206).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 206 to 209 is executed for allof the information 108 of the unique identification of the moving imageextracted from the other moving image data (in the step 210).

In case the digital moving image data 2601 is the compressed movingimage data 301, the following process (the steps 402 to 410) isexecuted.

The extracting unit 302 extracts the information 303 of the featurequantity of the compressed moving image, which includes the DCTcoefficients of each frame and the information of the motioncompensation of each frame, from the compressed moving image data 301(in the step 402).

Next, the extracting unit 304 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of the compressed movingimage from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the valuesand/or difference value of the values. On the other hand, the extractingunit 304 calculates values such as a motion vector, a mean motion vectorbetween frames before and behind, a total motion vector, a differencevector and/or a motion vector on a whole frame from the information ofthe motion compensation. And the extracting unit 304 arranges at leastone value from the above mentioned values on the time axis and thenpicks out all values arranged on the time axis, or some values from thevalues arranged on the time axis at uniform intervals or irregularintervals. And then, the extracting unit 304 extracts the picked outvalues as the information 305 of the unique identification of thecompressed moving image to be used for identifying the compressed movingimage (in the step 403).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 305of the unique identification of the compressed moving image extracted bythe extracting unit 304 (in the step 404).

On the other hand, the information 308 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 401 to 404 (in FIGS. 4A and 4B).

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 308 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209.is executed for allof the information 308 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

As mentioned above, according to the thirteenth embodiment of theinvention, whether the inputted digital moving image data 2601 is theuncompressed moving image data 101 or the compressed moving image data301 can be judged automatically. Consequently, identical moving imagecan be retrieved and identified from the moving image at the higherspeed regardless of the type (compressed/uncompressed) of image data tobe retrieved.

An apparatus and a method for retrieving moving image in the fourteenthpreferred embodiment according to the invention will be explained inFIGS. 28, 29A and 29B.

FIG. 28 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 28 withFIG. 5 and FIG. 7. In FIG. 28, the apparatus for retrieving moving imagecomprises an extracting unit 102 for extracting information 103 of thefeature quantity of moving image, which includes physical information ofthe feature quantity of moving image such as information of luminanceand color difference (Y, Cb, Cr) of each frame and/or information ofcolor (R, G, B) of each frame, from moving image data 101, a detectingunit 501 for detecting a cut point of the moving image data 101 asinformation 502 of a cut point, an extracting unit 503 for arranging atleast one physical information included in the information 103 (such asthe information of the luminance and the color difference (Y, Cb, Cr) ofeach frame and/or the information of the color (R, G, B) of each frame)on the time axis and for extracting information 505 of the uniqueidentification of moving image to identify the moving image in each shotunit of the moving image data 101 divided by using the information 502of the cut point from the arrangement of the physical information on thetime axis, a storing unit 106 for storing the information 505 of theunique identification of the moving image extracted by the extractingunit 503, a retrieving unit 109 for retrieving identical informationwith the information 505 of the unique identification of the movingimage stored in the storing unit 106 from information 508 of the uniqueidentification of moving image beforehand extracted from other movingimage data, an extracting unit 302 for extracting information 303 of thefeature quantity of compressed moving image, which includes DCT(Discrete Cosine Transform) coefficients of each frame and informationof motion compensation of each frame, from compressed moving image data301 provided by the standard compression encoding process for movingimage such as MPEG, a detecting unit 701 for detecting a cut point ofthe compressed moving image data 301 as information 702 of a cut point,an extracting unit 703 for arranging at least one information of featurequantity from the DCT coefficients of each frame and the information ofthe motion compensation of each frame, which are included in theinformation 303, on the time axis and for extracting information 705 ofthe unique identification of compressed moving image to identify thecompressed moving image from the arrangement of the information of thefeature quantity on the time axis in each shot unit of the compressedmoving image data 301 divided by using the information 702 of the cutpoint, a storing unit 306 for storing the information 705 of the uniqueidentification of the compressed moving image extracted by theextracting unit 703, a retrieving unit 309 for retrieving identicalinformation with the information 705 of the unique identification of thecompressed moving image stored in the storing unit 306 from information708 of the unique identification of compressed moving image beforehandextracted from other moving image data, a judging unit 2602 for judgingwhether inputted digital moving image data 2601 is the moving image data101 or the compressed moving image data 301 and for outputting themoving image data 101 to the extracting unit 102 and the compressedmoving image data 301 to the extracting unit 302, a storing unit 110 forstoring the retrieving result provided by the retrieving unit 109 and/orthe retrieving unit 309, and an output unit 111 for displaying and/orprinting the retrieving result stored in the storing unit 110.

FIGS. 29A and 29B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 28. Still, an identical step number is appended to the stephaving the identical process in FIGS. 29A and 29B with FIGS. 6A, 6B, 8Aand 8B. In FIGS. 28, 29A and 29B, at first the compressed/uncompressedjudging unit 2602 acquires the digital moving image data 2601 (in thestep 2701). Then, the compressed/uncompressed judging unit 2602 judgeswhether the acquired digital moving image data 2601 is the uncompressedmoving image data 101 or the compressed moving image data 301 (in thestep 2702).

In case the digital moving image data 2601 is the uncompressed movingimage data 101, the following process (the steps 202 to 210) isexecuted.

The extracting unit 102 extracts the information 103 of the featurequantity of the moving image, which includes the physical information ofthe feature quantity of the moving image such as the information of theluminance and the color difference (Y, Cb, Cr) of each frame and/or theinformation of the color (R, G, B) of each frame, from the moving imagedata 101 (in the step 202).

And, the detecting unit 501 detects the cut point of the moving imagedata 101 as the information 502 of the cut point (in the step 602). Thiscut point of the moving image data 101 shows a point where the scene ofthe moving image data 101 changes or a point where the camera forphotographing the moving image data 101 switches.

Next, the extracting unit 503 for extracting the information of theunique identification of the moving image arranges at least one valuefrom values of the physical information of the feature quantity of themoving image (such as the information of the luminance and the colordifference (Y, Cb, Cr) of each frame and/or the information of the color(R, G, B) of each frame) based on the information 103, mean value of thevalues, total value of the values and/or difference value of the valueson the time axis. Then, the extracting unit 503 picks out all valuesarranged on the time axis, or some values from the values arranged onthe time axis at uniform intervals or irregular intervals. And then, theextracting unit 503 extracts the picked out values in each shot unit ofthe moving image data 101 divided by the information 502 of the cutpoint as the information 505 of the unique identification of the movingimage to be used for identifying the moving image (in the step 603).

The storing unit 106 for storing the information of the uniqueidentification of the moving image stores the information 505 of theunique identification of the moving image extracted by the extractingunit 503 (in the step 604).

On the other hand, the information 508 of the unique identification ofthe moving image is extracted from other moving image data (the objectmoving image data for retrieving) (in the step 205). This extractingprocess can be executed by the processing equal to the foregoing steps601 to 604 (in FIGS. 6A and 6B).

Then, the retrieving unit 109 for retrieving the information of theunique identification of the moving image compares the information 505of the unique identification of the moving image stored in the storingunit 106 with the information 508 of the unique identification of themoving image extracted from the other moving image data (in the step206).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displayingand/or~printing the comparing result (in the step 208), it is outputtedby the output unit 111 for displaying and/or printing the retrievingresult (in the step 209).

The process of the above mentioned steps 206 to 209 is executed for allof the information 508 of the unique identification of the moving imageextracted from the other moving image data (in the step 210).

In case the digital moving image data 2601 is the compressed movingimage data 301, the following process (the steps 402 to 410) isexecuted.

The extracting unit 302 extracts the information 303 of the featurequantity of the compressed moving image, which includes the DCTcoefficients of each frame and the information of the motioncompensation of each frame, from the compressed moving image data 301(in the step 402). And, the detecting unit 701 detects the cut point ofthe compressed moving image data 301 as the information 702 of the cutpoint (in the step 802).

Next, the extracting unit 703 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of the compressed movingimage from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the valuesand/or difference value of the values. On the other hand, the extractingunit 703 calculates values such as a motion vector, a mean motion vectorbetween frames before and behind, a total motion vector, a differencevector and/or a motion vector on a whole frame from the information ofthe motion compensation. And the extracting unit 703 arranges at leastone value from the above mentioned values on the time axis and thenpicks out all values arranged on the time axis, or some values from thevalues arranged on the time axis at uniform intervals or irregularintervals. And then, the extracting unit 703 extracts the picked outvalues in each shot unit of the compressed moving image data 301 dividedby using the information 702 of the cut point as the information 705 ofthe unique identification of the compressed moving image to be used foridentifying the compressed moving image (in the step 803).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 705of the unique identification of the compressed moving image extracted bythe extracting unit 703 (in the step 804).

On the other hand, the information 708 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 801 to 804 (in FIGS. 8A and 8B).

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 708 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 708 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

As mentioned above, according to the fourteenth embodiment of theinvention, whether the inputted digital moving image data 2601 is theuncompressed moving image data 101 or the compressed moving image data301 can be judged automatically. Consequently, identical moving imagecan be retrieved and identified from the moving image at the higherspeed regardless of the type (compressed/uncompressed) of retrievingimage data.

An apparatus and a method for retrieving moving image in the fifteenthpreferred embodiment according to the invention will be explained inFIGS. 30, 31A and 31B.

FIG. 30 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 30 withFIG. 9. In FIG. 30, the apparatus for retrieving moving image comprisesan extracting unit 901 for extracting information of motion compensationin the M×N (M and N are natural numbers) pixels unit of moving imagedata 101, a DCT (Discrete Cosine Transform) operation unit 902 forobtaining a DCT coefficient by executing a DCT operation in the M′×N′(M′ and N′ are natural numbers) pixels unit of the moving image data101, an extracting unit 302 for extracting information 303 of thefeature quantity of compressed moving image, which includes DCTcoefficients of each frame and information of motion compensation ofeach frame, from compressed moving image data 301 provided by thestandard compression encoding process for moving image such as MPEG, anextracting unit 304 for arranging at least one information of featurequantity from the DCT coefficients of each frame and the information ofthe motion compensation of each frame, which are the information 303extracted by the extracting unit 302, or by the extracting unit 901 andthe DCT operation unit 902, on the time axis and for extractinginformation 305 of the unique identification of compressed moving imageto identify the compressed moving image from the arrangement of theinformation of feature quantity on the time axis, a storing unit 306 forstoring the information 305 of the unique identification of thecompressed moving image extracted by the extracting unit 304, aretrieving unit 309 for retrieving identical information with theinformation 305 of the unique identification of the compressed movingimage stored in the storing unit 306 from information 308 of the uniqueidentification of compressed moving image beforehand extracted fromother moving image data, a storing unit 110 for storing the retrievingresult provided by the retrieving unit 309, an output unit 111 fordisplaying and/or printing the retrieving result stored in the storingunit 110, and a judging unit 2602 for judging whether inputted digitalmoving image data 2601 is the moving image data 101 or the compressedmoving image data 301 and for outputting the moving image data 101 tothe extracting unit 102 and the compressed moving image data 301 to theextracting unit.

FIGS. 31A and 31B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 30. Still, an identical step number is appended to the stephaving the identical process in FIGS. 31A and 31B with FIGS. 10A and10B. In FIGS. 30, 31A and 31B, at first the compressed/uncompressedjudging unit 2602 acquires the digital moving image data 2601 (in thestep 2701). Then, the compressed/uncompressed judging unit 2602 judgeswhether the acquired digital moving image data 2601 is the uncompressedmoving image data 101 or the compressed moving image data 301 (in thestep 2702).

In case the digital moving image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputsthe moving image data 101 to the extracting unit 901 for extracting theinformation of the motion compensation and the DCT operation unit 902.And, the following process (the steps 1003 to 410) is executed.

The extracting unit 901 divides each frame of the moving image data 101into the M×N (M and N are natural numbers) pixels units, then calculatesthe information of the motion compensation in the M×N pixels unit (inthe step 1003). On the other hand, the DCT operation unit 902 divideseach frame of the moving image data 101 into the M′×N′ (M′ and N′ arenatural numbers) pixels units, then calculates the DCT coefficient byexecuting the DCT operation in the M′×N′ pixels unit (in the step 1004).

On the other hand, in case the digital moving image data 2601 is thecompressed moving image data 301, the compressed/uncompressed judgingunit 2602 outputs the compressed moving image data 301 to the extractingunit 302 for extracting the information of the feature quantity of thecompressed moving image. And, the following process (the steps 402 to410) is executed.

The extracting unit 302 extracts the information 303 of the featurequantity of the compressed moving image, which includes the DCTcoefficients of each frame and the information of the motioncompensation of each frame, from the compressed moving image data 301(in the step 402).

Next, the extracting unit 304 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of the compressed movingimage from the DCT coefficients of each frame, which is included in theinformation 303 extracted by extracting unit 302, or the extracting unit901 and the DCT operation unit 902, mean value of the values, totalvalue of the values and/or difference value of the values. On the otherhand, the extracting unit 304 calculates values such as a motion vector,a mean motion vector between frames before and behind, a total motionvector, a difference vector and/or a motion vector on a whole frame fromthe information of the motion compensation. And the extracting unit 304arranges at least one value from the above mentioned values on the timeaxis and then picks out all values arranged on the time axis, or somevalues from the values arranged on the time axis at uniform intervals orirregular intervals. And then, the extracting unit 304 extracts thepicked out values as the information 305 of the unique identification ofthe compressed moving image to be used for identifying the compressedmoving image (in the step 403).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 305of the unique identification of the compressed moving image extracted bythe extracting unit 304 (in the step 404).

On the other hand, the information 308 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 404 (in FIGS. 10A and 10B).

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 308 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 308 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

As mentioned above, according to the fifteenth embodiment of theinvention, whether the inputted digital moving image data 2601 is theuncompressed moving image data 101 or the compressed moving image data301 can be judged automatically. Consequently, identical moving imagecan be retrieved and identified from the moving image at the higherspeed regardless of the type (compressed/uncompressed) of retrievingimage data.

An apparatus and a method for retrieving moving image in the sixteenthpreferred embodiment according to the invention will be explained inFIGS. 32, 33A and 33B.

FIG. 32 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 32 withFIG. 11. In FIG. 32, the apparatus for retrieving moving image comprisesan extracting unit 901 for extracting information of motion compensationin the M×N (M and N are natural numbers) pixels unit of moving imagedata 101, a DCT (Discrete Cosine Transform) operation unit 902 forobtaining a DCT coefficient by executing a DCT operation in the M′×N′(M′ and N′ are natural numbers) pixels unit of the moving image data101, a detecting unit 501 for detecting a cut point of the moving imagedata 101 as information 502 of a cut point, an extracting unit 302 forextracting information 303 of the feature quantity of compressed movingimage, which includes DCT coefficients of each frame and information ofmotion compensation of each frame, from compressed moving image data 301provided by the standard compression encoding process for moving imagesuch as MPEG, a detecting unit 701 for detecting a cut point of thecompressed moving image data 301 as information 702 of a cut point, anextracting unit 703 for arranging at least one information of featurequantity from the DCT coefficients of each frame and the information ofthe motion compensation of each frame, which are included in theinformation 303 extracted by the extracting unit 901 and the DCToperation unit 902, or the extracting unit 302, on the time axis and forextracting information 705 of the unique identification of compressedmoving image to identify the compressed moving image from thearrangement of the information of the feature quantity on the time axisin each shot unit of the compressed moving image data 301 divided byusing the information 702 of the cut point, a storing unit 306 forstoring the information 705 of the unique identification of thecompressed moving image extracted by the extracting unit 703, aretrieving unit 309 for retrieving identical information with theinformation 705 of the unique identification of the compressed movingimage stored in the storing unit 306 from information 708 of the uniqueidentification of compressed moving image beforehand extracted fromother moving image data, a storing unit 110 for storing the retrievingresult provided by the retrieving unit 309, an output unit 111 fordisplaying and/or printing the retrieving result stored in the storingunit 110, and a judging unit 2602 for judging whether inputted digitalmoving image data 2601 is the moving image data 101 or the compressedmoving image data 301 and for outputting the moving image data 101 tothe extracting unit, the DCT (Discrete Cosine Transform) operation unit902 and the detecting unit, and the compressed moving image data 301 tothe extracting unit 302 and the detecting unit 701.

FIGS. 33A and 33B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 32. Still, an identical step number is appended to the stephaving the identical process in FIGS. 33A and 33B with FIGS. 12A and12B. In FIGS. 32, 33A and 33B, at first the compressed/uncompressedjudging unit 2602 acquires the digital moving image data 2601 (in thestep 2701). Then, the compressed/uncompressed judging unit 2602 judgeswhether the acquired digital moving image data 2601 is the uncompressedmoving image data 101 or the compressed moving image data 301 (in thestep 2702).

In case the digital moving image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputsthe moving image data 101 to the extracting unit 901 for extracting theinformation of the motion compensation, the DCT operation unit 902 andthe detecting unit 501 for detecting the cut point. And, the followingprocess (the steps 1003 to 410) is executed.

The extracting unit 901 divides each frame of the moving image data 101into the M×N (M and N are natural numbers) pixels units, then calculatesthe information of the motion compensation in the M×N pixels unit (inthe step 1003). On the other hand, the DCT operation unit 902 divideseach frame of the moving image data 101 into the M′×N′ (M′ and N′ arenatural numbers) pixels units, then calculates the DCT coefficient byexecuting the DCT operation in the M′×N′ pixels unit (in the step 1004).

And, the detecting unit 501 detects the cut point of the moving imagedata 101 as the information 502 of the cut point (in the step 602).

On the other hand, in case the digital moving image data 2601 is thecompressed moving image data 301, the compressed/uncompressed judgingunit 2602 outputs the compressed moving image data 301 to the extractingunit 302 for extracting the information of the feature quantity of thecompressed moving image and the detecting unit 701 for detecting the cutpoint. And, the following process (the steps 402 to 410) is executed.

The extracting unit 302 extracts the information 303 of the featurequantity of the compressed moving image, which includes the DCTcoefficients of each frame and the information of the motioncompensation of each frame, from the compressed moving image data 301(in the step 402). And, the detecting unit 701 detects the cut point ofthe compressed moving image data 301 as the information 702 of the cutpoint (in the step 802).

Next, the extracting unit 703 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of the compressed movingimage from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the valuesand/or difference value of the values. On the other hand, the extractingunit 703 calculates values such as a motion vector, a mean motion vectorbetween frames before and behind, a total motion vector, a differencevector and/or a motion vector on a whole frame from the information ofthe motion compensation. And the extracting unit 703 arranges at leastone value from the above mentioned values on the time axis and thenpicks out all values arranged on the time axis, or some values from thevalues arranged on the time axis at uniform intervals or irregularintervals. And then, the extracting unit 703 extracts the picked outvalues in each shot unit of the moving image data 101 divided by usingthe information 502 or in each shot unit of the compressed moving imagedata 301 divided by using the information 702 as the information 705 ofthe unique identification of the compressed moving image to be used foridentifying the compressed moving image (in the step 803).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 705of the unique identification of the compressed moving image extracted bythe extracting unit 703 (in the step 804).

On the other hand, the information 708 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 804 (in FIGS. 12A and 12B).

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 708 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 708 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

As mentioned above, according to the sixteenth embodiment of theinvention, whether the inputted digital moving image data 2601 is theuncompressed moving image data 101 or the compressed moving image data301 can be judged automatically. Consequently, identical moving imagecan be retrieved and identified from the moving image at the higherspeed regardless of the type (compressed/uncompressed) of retrievingimage data.

An apparatus and a method for retrieving moving image in the seventeenthpreferred embodiment according to the invention will be explained inFIGS. 34, 35A and 35B.

FIG. 34 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 34 withFIG. 13. In FIG. 34, the apparatus for retrieving moving image comprisesan extracting unit 901 for extracting information of motion compensationin the M×N (M and N are natural numbers) pixels unit of moving imagedata 101, an extracting unit 1301 for calculating an average pixelvalue, such as information of average color (R, G, B), and/orinformation of average luminance and average color difference (Y, Cb,Cr), in the M′×N′ (M′ and N′ are natural numbers) pixels unit, anextracting unit 1302 for extracting information 1303 of the featurequantity of compressed moving image, which includes DC (Direct Current)components of DCT (Discrete Cosine Transform) coefficients of each frameand information of motion compensation of each frame, from compressedmoving image data 301 provided by the standard compression encodingprocess for moving image such as MPEG, an extracting unit 1304 forarranging at least one information of feature quantity from the DCcomponents of the DCT coefficients of each frame and the information ofthe motion compensation of each frame or the average pixel value, whichare included in the information 1303 extracted by the extracting unit901 and the extracting unit 1301, or the extracting unit 1302, on thetime axis and for extracting information 305 of the uniqueidentification of compressed moving image to identify the compressedmoving image from the arrangement of the information of the featurequantity on the time axis, a storing unit 306 for storing theinformation 305 of the unique identification of the compressed movingimage extracted by the extracting unit 1304, a retrieving unit 309 forretrieving identical information with the information 305 of the uniqueidentification of the compressed moving image stored in the storing unit306 from information 308 of the unique identification of compressedmoving image beforehand extracted from other moving image data, astoring unit 110 for storing the retrieving result provided by theretrieving unit 309, an output unit 111 for displaying and/or printingthe retrieving result stored in the storing unit 110, and a judging unit2602 for judging whether inputted digital moving image data 2601 is themoving image data 101 or the compressed moving image data 301 and foroutputting the moving image data 101 to the extracting unit 901 and theextracting unit 1301, and the compressed moving image data 301 to theextracting unit 1302.

FIGS. 35A and 35B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 34. Still, an identical step number is appended to the stephaving the identical process in FIGS. 35A and 35B with FIGS. 15A and15B. In FIGS. 34, 35A and 35B, at first the compressed/uncompressedjudging unit 2602 acquires the digital moving image data 2601 (in thestep 2701). Then, the compressed/uncompressed judging unit 2602 judgeswhether the acquired digital moving image data 2601 is the uncompressedmoving image data 101 or the compressed moving image data 301 (in thestep 2702).

In case the digital moving image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputsthe moving image data 101 to the extracting unit 901 for extracting theinformation of the motion compensation and the extracting unit 1301 forextracting the average value of the block unit. And, the followingprocess (the steps 1003 to 410) is executed.

The extracting unit 901 divides each frame of the moving image data 101into the M×N (M and N are natural numbers) pixels units, then calculatesthe information of the motion compensation in the M×N pixels unit (inthe step 1003). On the other hand, the extracting unit 1301 divides eachframe of the moving image data 101 into the M′×N′ (M′ and N′ are naturalnumbers) pixels units, then calculates the average pixel value in theM′×N′ pixels unit (in the step 1502).

On the other hand, in case the digital moving image data 2601 is thecompressed moving image data 301, the compressed/uncompressed judgingunit 2602 outputs the compressed moving image data 301 to the extractingunit 1302 for extracting the information of the feature quantity of thecompressed moving image. And, the following process (the steps 1503 to410) is executed.

The extracting unit 1302 extracts the information 1303 of the featurequantity of the compressed moving image, which includes the DCcomponents of the DCT coefficients of each frame and the information ofthe motion compensation of each frame, from the compressed moving imagedata 301 (in the step 1503).

Next, the extracting unit 1304 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of compressed movingimage from the DC components of the DCT coefficients of each frameand/or values of the average pixel value of each frame, which areincluded in the information 1303, mean value of the values, total valueof the values and/or difference value of the values. On the other hand,the extracting unit 1304 calculates values such as a motion vector, amean motion vector between frames before and behind, a total motionvector, a difference vector and/or a motion vector on a whole frame fromthe information of the motion compensation. And the extracting unit 1304arranges at least one value from the above mentioned values on the timeaxis and then picks out all values arranged on the time axis, or somevalues from the values arranged on the time axis at uniform intervals orirregular intervals. And then, the extracting unit 1304 extracts thepicked out values as the information 305 of the unique identification ofthe compressed moving image to be used for identifying the compressedmoving image (in the step 1504).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 305of the unique identification of the compressed moving image extracted bythe extracting unit 1304 (in the step 404).

On the other hand, the information 308 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 404 (in FIGS. 15A and 15B).

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 308 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 308 of the unique identification of the compressedmoving image *99 extracted from the other moving image data (in the step410).

As mentioned above, according to the seventeenth embodiment of theinvention, whether the inputted digital moving image data 2601 is theuncompressed moving image data 101 or the compressed moving image data301 can be judged automatically. Consequently, identical moving imagecan be retrieved and identified from the moving image at the higherspeed regardless of the type (compressed/uncompressed) of retrievingimage data.

An apparatus and a method for retrieving moving image in the eighteenthpreferred embodiment according to the invention will be explained inFIGS. 36, 37A and 37B.

FIG. 36 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 36 withFIG. 16. In FIG. 36, the apparatus for retrieving moving image comprisesan extracting unit 901 for extracting information of motion compensationin the M×N (M and N are natural numbers) pixels unit of moving imagedata 101, an extracting unit 1301 for calculating an average pixel valuesuch as information of average color (R, G, B) and/or information ofaverage luminance and average color difference (Y, Cb, Cr) in the M′×N′(M′ and N′ are natural numbers) pixels unit, a detecting unit 501 fordetecting a cut point of the moving image data 101 as information 502 ofa cut point, an extracting unit 1302 for extracting information 1303 ofthe feature quantity of compressed moving image, which includes DC(Direct Current) components of DCT (Discrete Cosine Transform)coefficients of each frame and information of motion compensation ofeach frame, from compressed moving image data 301 provided by thestandard compression encoding process for moving image such as MPEG, adetecting unit 701 for detecting a cut point of the compressed movingimage data 301 as information 702 of a cut point, an extracting unit1603 for arranging at least one information of feature quantity from theDC components of the DCT coefficients of each frame and the informationof the motion compensation of each frame or the average pixel values,which are included in the information 1303 extracted by the extractingunit 901 and the extracting unit 1301, or the extracting unit 1302, onthe time axis and for extracting information 705 of the uniqueidentification of compressed moving image to identify the compressedmoving image from the arrangement of the information of the featurequantity on the time axis in each shot unit of the moving image data 101divided by using the information 502 of the cut point or in each shotunit of the compressed moving image data 301 divided by using theinformation 702 of the cut point, a storing unit 306 for storing theinformation 705 of the unique identification of the compressed movingimage extracted by the extracting unit 1603, a retrieving unit 309 forretrieving identical information with the information 705 of the uniqueidentification of the compressed moving image stored in the storing unit306 from information 708 of the unique identification of compressedmoving image beforehand extracted from other moving image data, astoring unit 110 for storing the retrieving result provided by theretrieving unit 309, an output unit 111 for displaying and/or printingthe retrieving result stored in the storing unit 110, and a judging unit2602 for judging whether inputted digital moving image data 2601 is themoving image data 101 or the compressed moving image data 301 and foroutputting the moving image data 101 to the extracting unit 901, theextracting unit 1301 and the detecting unit 501, and the compressedmoving image data 301 to the extracting unit 1302 and the detecting unit701.

FIGS. 37A and 37B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 36. Still, an identical step number is appended to the stephaving the identical process in FIGS. 37A and 37B with FIGS. 17A and17B. In FIGS. 36, 37A and 37B, at first the compressed/uncompressedjudging unit 2602 acquires the digital moving image data 2601 (in thestep 2701). Then, the compressed/uncompressed judging unit 2602 judgeswhether the acquired digital moving image data 2601 is the uncompressedmoving image data 101 or the compressed moving image data 301 (in thestep 2702).

In case the digital moving image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputsthe moving image data 101 to the extracting unit 901 for extracting theinformation of the motion compensation, the extracting unit 1301 forextracting the average value of the block unit and the detecting unit501 for detecting the cut point. And, the following process (the steps1003 to 410) is executed.

The extracting unit 901 divides each frame of the moving image data 101into the M×N (M and N are natural numbers) pixels units, then calculatesthe information of the motion compensation in the M×N pixels unit (inthe step 1003). And, the extracting unit 1301 divides each frame of themoving image data 101 into the M′×N′ (M′ and N′ are natural numbers)pixels units, then calculates the average pixel value in the M′×N′pixels unit (in the step 1502). And more, the detecting unit 501 detectsthe cut point of the moving image data 101 as the information 502 of thecut point (in the step 602).

On the other hand, in case the digital moving image data 2601 is thecompressed moving image data 301, the compressed/uncompressed judgingunit 2602 outputs the compressed moving image data 301 to the extractingunit 1302 for extracting the information of the feature quantity of thecompressed moving image and the detecting unit 701 for detecting the cutpoint. And, the following process (the steps 1503 to 410) is executed.

The extracting unit 1302 extracts the information 1303 of the featurequantity of the compressed moving image, which includes the DCcomponents of the DCT coefficients of each frame and the information ofthe motion compensation of each frame, from the compressed moving imagedata 301 (in the step 1503). And then, the detecting unit 701 detectsthe cut point of the compressed moving image data 301 as the informationof the cut point 702 (in the step 802).

Next, the extracting unit 1603 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of the compressed movingimage from the DC components of the DCT coefficients of each frameand/or values of the average pixel value of each frame, which areincluded in the information 1303, mean value of the values, total valueof the values and/or difference value of the values. On the other hand,the extracting unit 1603 calculates values such as a motion vector, amean motion vector between frames before and behind, a total motionvector, a difference vector and/or a motion vector on a whole frame fromthe information of the motion compensation. And the extracting unit 1603arranges at least one value from the above mentioned values on the timeaxis and then picks out all values arranged on the time axis, or somevalues from the values arranged on the time axis at uniform intervals orirregular intervals. And then, the extracting unit 1603 extracts thepicked out values in each shot unit of the moving image data 101 dividedby using the information 502 of the cut point or in each shot unit ofthe compressed moving image data 301 divided by using the information702 of the cut point as the information 705 of the unique identificationof the compressed moving image to be used for identifying the compressedmoving image (in the step 1702).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 705of the unique identification of the compressed moving image extracted bythe extracting unit 1603 (in the step 804).

On the other hand, the information 708 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 804 (in FIGS. 17A and 17B).

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 708 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 708 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

As mentioned above, according to the eighteenth embodiment of theinvention, whether the inputted digital moving image data 2601 is theuncompressed moving image data 101 or the compressed moving image data301 can be judged automatically. Consequently, identical moving imagecan be retrieved and identified from the moving image at the higherspeed regardless of the type (compressed/uncompressed) of retrievingimage data.

An apparatus and a method for retrieving moving image in the nineteenthpreferred embodiment according to the invention will be explained inFIGS. 38, 39A and 39B.

FIG. 38 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 38 withFIG. 18. In FIG. 38, the apparatus for retrieving moving image comprisesa DCT (.Discrete Cosine Transform) operation unit 902 for obtaining aDCT coefficient by executing a DCT operation in the M×N (M and N arenatural numbers) pixels unit of moving image data 101 as information 303of the feature quantity of compressed moving image, an extracting unit302 for extracting information 303 of the feature quantity of compressedmoving image, which includes DCT coefficients of each frame, fromcompressed moving image data 301 provided by the standard compressionencoding process for moving image such as MPEG, an extracting unit 304for arranging at least one information of feature quantity from the DCTcoefficients of each frame, which is included in the information 303extracted by the DCT operation unit 902 or by the extracting unit 302,on the time axis and for extracting information 305 of the uniqueidentification of compressed moving image to identify the compressedmoving image from the arrangement of the information of feature quantityon the time axis, a storing unit 306 for storing the information 305 ofthe unique identification of the compressed moving image extracted bythe extracting unit 304, a retrieving unit 309 for retrieving identicalinformation with the information 305 of the unique identification of thecompressed moving image stored in the storing unit 306 from information308 of the unique identification of compressed moving image beforehandextracted from other moving image data, a storing unit 110 for storingthe retrieving result provided by the retrieving unit 309, an outputunit 111 for displaying and/or printing the retrieving result stored inthe storing unit 110, and a judging unit 2602 for judging whetherinputted digital moving image data 2601 is the moving image data 101 orthe compressed moving image data 301 and for outputting the moving imagedata 101 to the DCT (Discrete Cosine Transform) operation unit 902 andthe compressed moving image data 301 to the extracting unit 302.

FIGS. 39A and 39B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 38. Still, an identical step number is appended to the stephaving the identical process in FIGS. 39A and 39B with FIGS. 19A and19B. In FIGS. 38, 39A and 39B, at first the compressed/uncompressedjudging unit 2602 acquires the digital moving image data 2601 (in thestep 2701). Then, the compressed/uncompressed judging unit 2602 judgeswhether the acquired digital moving image data 2601 is the uncompressedmoving image data 101 or the compressed moving image data 301 (in thestep 2702).

In case the digital moving image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputsthe moving image data 101 to the DCT operation unit 902. The followingprocess (the steps 1902 to 410) is executed.

The DCT operation unit 902 divides each frame of the moving image data101 into the M×N (M and N are natural numbers) pixels units, thencalculates the DCT coefficient by executing the DCT operation in the M×Npixels unit as the information 303 of the feature quantity of thecompressed moving image (in the step 1902).

On the other hand, in case the digital moving image data 2601 is thecompressed moving image data 301, the compressed/uncompressed judgingunit 2602 outputs the compressed moving image data 301 to the extractingunit 302 for extracting the information of the feature quantity of thecompressed moving image. The following process (the steps 1903 to 410)is executed.

The extracting unit 302 extracts the information 303 of the featurequantity of the compressed moving image, which includes the DCTcoefficients of each frame, from the compressed moving image data 301 as(in the step 1903).

Next, the extracting unit 304 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of the compressed movingimage from the DCT coefficients of each frame, which is included in theinformation 303 extracted by the extracting unit 902, mean value of thevalues, total value of the values and/or difference value of the values.And the extracting unit 304 arranges at least one value from the abovementioned values on the time axis and then picks out all values arrangedon the time axis, or some values from the values arranged on the timeaxis at uniform intervals or irregular intervals. And then, theextracting unit 304 extracts the picked out values as the information305 of the unique identification of the compressed moving image to beused for identifying the compressed moving image (in the step 1994).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 305of the unique identification of the compressed moving image extracted bythe extracting unit 304 (in the step 404).

On the other hand, the information 308 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 404 (in FIGS. 19A and 19B).

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 308 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 308 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

As mentioned above, according to the nineteenth embodiment of theinvention, whether the inputted digital moving image data 2601 is theuncompressed moving image data 101 or the compressed moving image data301 can be judged automatically. Consequently, identical moving imagecan be retrieved and identified from the moving image at the higherspeed regardless of the type (compressed/uncompressed) of retrievingimage data.

An apparatus and a method for retrieving moving image in the twentiethpreferred embodiment according to the invention will be explained inFIGS. 40, 41A and 41B.

FIG. 40 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 40 withFIG. 20. In FIG. 40, the apparatus for retrieving moving image comprisesa DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCTcoefficient as information 303 of the feature quantity of compressedmoving image by executing a DCT operation in the M×N (M and N arenatural numbers) pixels unit of moving image data 101, a detecting unit501 for detecting a cut point of the moving image data 101 asinformation 502 of a cut point, an extracting unit 302 for extractinginformation 303 of the feature quantity of compressed moving image,which includes DCT coefficients of each frame, from compressed movingimage data 301 provided by the standard compression encoding process formoving image such as MPEG, a detecting unit 701 for detecting a cutpoint of the compressed moving image data 301 as information 702 of acut point, an extracting unit 703 for arranging at least one informationof feature quantity from the DCT coefficients of each frame, which isincluded in the information 303 extracted by the DCT operation unit 902or the extracting unit 302, on the time axis and for extractinginformation 705 of the unique identification of compressed moving imageto identify the compressed moving image from the arrangement of theinformation of the feature quantity on the time axis in each shot unitof the compressed moving image data 301 divided by using the information502 of the cut point and/or the information 702 of the cut point, astoring unit 306 for storing the information 705 of the uniqueidentification of the compressed moving image extracted by theextracting unit 703, a retrieving unit 309 for retrieving identicalinformation with the information 705 of the unique identification of thecompressed moving image stored in the storing unit 306 from information708 of the unique identification of compressed moving image beforehandextracted from other moving image data, a storing unit 110 for storingthe retrieving result provided by the retrieving unit 309, an outputunit 111 for displaying and/or printing the retrieving result stored inthe storing unit 110, and a judging unit 2602 for judging whetherinputted digital moving image data 2601 is the moving image data 101 orthe compressed moving image data 301 and for outputting the moving imagedata 101 to the DCT (Discrete Cosine Transform) operation unit 902 andthe detecting unit 501, and the compressed moving image data 301 to theextracting unit 302 and the detecting unit 701.

FIGS. 41A and 41B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 40. Still, an identical step number is appended to the stephaving the identical process in FIGS. 41A and 41B with FIGS. 21A and21B. In FIGS. 40, 41A and 41B, at first the compressed/uncompressedjudging unit 2602 acquires the digital moving image data 2601 (in thestep 2701). Then, the compressed/uncompressed judging unit 2602 judgeswhether the acquired digital moving image data 2601 is the uncompressedmoving image data 101 or the compressed moving image data 301 (in thestep 2702).

In case the digital moving image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputsthe moving image data 101 to the DCT operation unit 902 and thedetecting unit 501 for detecting the cut point. And, the followingprocess (the steps 1902 to 410) is executed.

The DCT operation unit 902 divides each frame of the moving image data101 into the M×N (M and N are natural numbers) pixels units, thencalculates the DCT coefficient by executing the DCT operation in the M×Npixels unit as the information 303 of the feature quantity of thecompressed moving image (in the step 1902). On the other hand, thedetecting unit 501 for detecting the cut point detects the cut point ofthe moving image data 101 as the information 502 of the cut point (inthe step 602).

On the other hand, in case the digital moving image data 2601 is thecompressed moving image data 301, the compressed/uncompressed judgingunit 2602 outputs the compressed moving image data 301 to the extractingunit 302 for extracting the information of the feature quantity of thecompressed moving image and the detecting unit 701 for detecting the cutpoint. The following process (the steps 1903 to 410) is executed.

The extracting unit 302 extracts the information 303 of the featurequantity of the compressed moving image, which includes the DCTcoefficients of each frame, from the compressed moving image data 301(in the step 1903). And, the detecting unit 701 detects the cut point ofthe compressed moving image data 301 as the information 702 of the cutpoint (in the step 802).

Next, the extracting unit 703 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of the compressed movingimage from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the valuesand/or difference value of the values. And the extracting unit 703arranges at least one value from the above mentioned values on the timeaxis and then picks out all values arranged on the time axis, or somevalues from the values arranged on the time axis at uniform intervals orirregular intervals. And then, the extracting unit 703 extracts thepicked out values in each shot unit of the moving image data 101 dividedby using the information 502 or in each shot unit of the compressedmoving image data 301 divided by using the information 702 as theinformation 705 of the unique identification of the compressed movingimage to be used for identifying the compressed moving image (in thestep 2102).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 705of the unique identification of the compressed moving image extracted bythe extracting unit 703 (in the step 804).

On the other hand, the information 708 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 804 (in FIGS. 21A and 21B).

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 708 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 708 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

As mentioned above, according to the twentieth embodiment of theinvention, whether the inputted digital moving image data 2601 is theuncompressed moving image data 101 or the compressed moving image data301 can be judged automatically. Consequently, identical moving imagecan be retrieved and identified from the moving image at the higherspeed regardless of the type (compressed/uncompressed) of retrievingimage data.

An apparatus and a method for retrieving moving image in thetwenty-first preferred embodiment according to the invention will beexplained in FIGS. 42, 43A and 43B.

FIG. 42 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 42 withFIG. 22. In FIG. 42, the apparatus for retrieving moving image comprisesan extracting unit 1301 for calculating an average pixel value, such asinformation of average color (R, G, B), and/or information of averageluminance and average color difference (Y, Cb, Cr), in the M×N (M and Nare natural numbers) pixels unit, an extracting unit 1302 for extractinginformation 1303 of the feature quantity of compressed moving image,which includes DC (Direct Current) components of DCT (Discrete CosineTransform) coefficients of each frame, from compressed moving image data301 provided by the standard compression encoding process for movingimage such as MPEG, an extracting unit 1304 for arranging at least oneinformation of feature quantity from the DC components of the DCTcoefficients of each frame or the average pixel value, which is includedin the information 1303 extracted by the extracting unit 1301 and/or theextracting unit 1302, on the time axis and for extracting information305 of the unique identification of compressed moving image to identifycompressed moving image from the arrangement of the information of thefeature quantity on the time axis, a storing unit 306 for storing theinformation 305 of the unique identification of the compressed movingimage extracted by the extracting unit 1304, a retrieving unit 309 forretrieving identical information with the information 305 of the uniqueidentification of the compressed moving image stored in the storing unit306 from information 308 of the unique identification of compressedmoving image beforehand extracted from other moving image data, astoring unit 110 for storing the retrieving result provided by theretrieving unit 309, an output unit 111 for displaying and/or printingthe retrieving result stored in the storing unit 110, and a judging unit2602 for judging whether inputted digital moving image data 2601 is themoving image data 101 or the compressed moving image data 301 and foroutputting the moving image data 101 to the extracting unit 1301 and thecompressed moving image data 301 to the extracting unit 1302.

FIGS. 43A and 43B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 42. Still, an identical step number is appended to the stephaving the identical process in FIGS. 43A and 43B with FIGS. 23A and23B. In FIGS. 42, 43A and 43B, at first the compressed/uncompressedjudging unit 2602 acquires the digital moving image data 2601 (in thestep 2701). Then, the compressed/uncompressed judging unit 2602 judgeswhether the acquired digital moving image data 2601 is the uncompressedmoving image data 101 or the compressed moving image data 301 (in thestep 2702).

In case the digital moving image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputsthe moving image data 101 to the extracting unit 1301 for extracting theaverage value of the block unit. The following process (the steps 1502to 410) is executed.

The extracting unit 1301 divides each frame of the moving image data 101into the M×N (M and N are natural numbers) pixels units, then calculatesthe average pixel value in the M×N pixels unit as the information 1303of the feature quantity of the compressed moving image (in the step1502).

On the other hand, in case the digital moving image data 2601 is thecompressed moving image data 301, the compressed/uncompressed judgingunit 2602 outputs the compressed moving image data 301 to the extractingunit 1302 for extracting the information of the feature quantity of thecompressed moving image. The following process (the steps 1503 to 410)is executed.

The extracting unit 1302 extracts the information 1303 of the featurequantity of the compressed moving image, which includes the DCcomponents of the DCT coefficients of each frame, from the compressedmoving image data 301 (in the step 1503).

Next, the extracting unit 1304 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of compressed movingimage from the DC components of the DCT coefficients of each frameand/or values of the average pixel value of each frame, which areincluded in the information 1303, mean value of the values, total valueof the values and/or difference value of the values. And the extractingunit 1304 arranges at least one value from the above mentioned values onthe time axis and then picks out all values arranged on the time axis,or some values from the values arranged on the time axis at uniformintervals or irregular intervals. And then, the extracting unit 1304extracts the picked out values as the information 305 of the uniqueidentification of the compressed moving image to be used for identifyingthe compressed moving image (in the step 1504).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 305of the unique identification of the compressed moving image extracted bythe extracting unit 1304 (in the step 404).

On the other hand, the information 308 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 404 (in FIGS. 23A and 23B).

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 308 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 308 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

As mentioned above, according to the twenty-first embodiment of theinvention, whether the inputted digital moving image data 2601 is theuncompressed moving image data 101 or the compressed moving image data301 can be judged automatically. Consequently, identical moving imagecan be retrieved and identified from the moving image at the higherspeed regardless of the type (compressed/uncompressed) of retrievingimage data.

An apparatus and a method for retrieving moving image in thetwenty-second preferred embodiment according to the invention will beexplained in FIGS. 44, 45A and 45B.

FIG. 44 is a block diagram showing an apparatus for retrieving movingimage according to the present invention. Still, an identical code isappended to the thing having the identical composition in FIG. 44 withFIG. 24. In FIG. 44, the apparatus for retrieving moving image comprisesan extracting unit 1301 for calculating an average pixel value such asinformation of average color (R, G, B) 5 and/or information of averageluminance and average color difference (Y, Cb, Cr) in the M×N (M and Nare natural numbers) pixels unit of moving image data 101 as information1303 of the feature quantity of compressed moving image, a detectingunit 501 for detecting a cut point of the moving image data 101 asinformation 502 of a cut point, an extracting unit 1302 for extracting10 information 1303 of the feature quantity of compressed moving image,which includes DC (Direct Current) components of DCT (Discrete CosineTransform) coefficients of each frame, from compressed moving image data301 provided by the standard compression encoding process for movingimage such as MPEG, a detecting unit 701 for detecting a cut point ofthe compressed moving image data 15 301 as information 702 of a cutpoint, an extracting unit 1603 for arranging at least one information offeature quantity from the DC components of the DCT coefficients of eachframe or the average pixel value, which is included in the information1303 extracted by the extracting unit 1301 and/or the extracting unit1302, on the time axis and for extracting information 705 of the unique20 identification of compressed moving image to identify the compressedmoving image from the arrangement of the information of the featurequantity on the time axis in each shot unit of the moving image data 101divided by using the information 502 of the cut point or in each shotunit of the compressed moving image data 301 divided by using theinformation 702 of the cut point, a storing 25 unit 306 for storing theinformation 705 of the unique identification of the compressed movingimage extracted by the extracting unit 1603, a retrieving unit 309 forretrieving identical information with the information 705 of the uniqueidentification of the compressed moving image stored in the storing unit306 from information 708 of the unique identification of compressedmoving image beforehand extracted from other moving image data, astoring unit 110 for storing the retrieving result provided by theretrieving unit 309, an output unit 111 for displaying and/or printingthe retrieving result stored in the storing unit 110, and a judging unit2602 for judging whether inputted digital moving image data 2601 is themoving image data 101 or the compressed moving image data 301 and foroutputting the moving image data 101 to the extracting unit and thedetecting unit 501, and the compressed moving image data 301 to theextracting unit 1302 and the detecting unit 701.

FIGS. 45A and 45B are flowcharts of the process in the apparatus forretrieving the moving image according to the present invention describedin FIG. 44. Still, an identical step number is appended to the stephaving the identical process in FIGS. 45A and 45B with FIGS. 25A and25B. In FIGS. 44, 45A and 45B, at first the compressed/uncompressedjudging unit 2602 acquires the digital moving image data 2601 (in thestep 2701). Then, the compressed/uncompressed judging unit 2602 judgeswhether the acquired digital moving image data 2601 is the uncompressedmoving image data 101 or the compressed moving image data 301 (in thestep 2702).

In case the digital moving image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputsthe moving image data 101 to the extracting unit 1301 for extracting theaverage value of the block unit and the detecting unit 501 for detectingthe cut point. And, the following process (the steps 1502 to 410) isexecuted.

The extracting unit 1301 divides each frame of the moving image data 101into the M×N (M and N are natural numbers) pixels units, then calculatesthe average pixel value in the M×N pixels unit (in the step 1502). And,the detecting unit 501 detects the cut point of the moving image data101 as the information 502 of the cut point (in the step 602).

On the other hand, in case the digital moving image data 2601 is thecompressed moving image data 301, the compressed/uncompressed judgingunit 2602 outputs the compressed moving image data 301 to the extractingunit 1302 for extracting the information of the feature quantity of thecompressed moving image and the detecting unit 701 for detecting the cutpoint. And, the following process (the steps 1503 to 410) is executed.

The extracting unit 1302 extracts the information 1303 of the featurequantity of the compressed moving image, which includes the DCcomponents of the DCT coefficients of each frame, from the compressedmoving image data 301 (in the step 1503). And, the detecting unit 701detects the cut point of the compressed moving image data 301 as theinformation 702 of the cut point (in the step 802).

Next, the extracting unit 1603 for extracting the information of theunique identification of the compressed moving image calculates valuesof physical information of the feature quantity of compressed movingimage from the DC components of the DCT coefficients of each frameand/or the average pixel value of each frame, which are included in theinformation 1303, mean value of the values, total value of the valuesand/or difference value of the values. And the extracting unit 1603arranges at least one value from the above mentioned values on the timeaxis and then picks out all values arranged on the time axis, or somevalues from the values arranged on the time axis at uniform intervals orirregular intervals. And then, the extracting unit 1603 extracts thepicked out values in each shot unit of the moving image data 101 dividedby using the information 502 of the cut point or in each shot unit ofthe compressed moving image data 301 divided by using the information702 of the cut point as the information 705 of the unique identificationof the compressed moving image to be used for identifying the compressedmoving image (in the step 1702).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information 705of the unique identification of the compressed moving image extracted bythe extracting unit 1603 (in the step 804).

On the other hand, the information 708 of the unique identification ofthe compressed moving image is extracted from other compressed movingimage data (the object compressed moving image data for retrieving) (inthe step 405). This extracting process can be executed by the processingequal to the foregoing steps 1001 to 804 (in FIGS. 25A and 25B).

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 708 of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 708 of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

As mentioned above, according to the twenty-second embodiment of theinvention, whether the inputted digital moving image data 2601 is theuncompressed moving image data 101 or the compressed moving image data301 can be judged automatically. Consequently, identical moving imagecan be retrieved and identified from the moving image at the higherspeed regardless of the type (compressed/uncompressed) of retrievingimage data.

An apparatus and a method for retrieving moving image in thetwenty-third preferred embodiment according to the invention will beexplained in FIGS. 46, 47A and 47B.

FIG. 46 is a block diagram showing an apparatus for monitoringtelevision commercials, which is a specific example of the secondembodiment according to the present invention. Still, an identical codeis appended to the thing having the identical composition in FIG. 46with FIG. 3.

In FIG. 46, the apparatus for monitoring television commercialscomprises an extracting unit 302A for extracting information (a DCcomponent) 303A of the feature quantity of compressed moving image,which includes the only DC (Direct Current) components of DCT (DiscreteCosine Transform) coefficients of each frame, from MPEG-2 stream(television broadcasting) 301A compressed by MPEG-2 process, anextracting unit (a calculating unit for calculating average colorinformation) 304A for arranging an average value of the DC components ofeach frame, which is included in the information 303A extracted by theextracting unit 302A, on the time axis and for extracting thearrangement of the average values on the time axis as information (timeseries information of average color information) 305A of the uniqueidentification of compressed moving image to identify the compressedmoving image, a storing unit 306 for storing the information 305A of theunique identification of the compressed moving image extracted by theextracting unit 304A, a retrieving unit 309 for retrieving identicalinformation with the information 305A of the unique identification ofthe compressed moving image stored in the storing unit 306 frominformation (time series information of average color information of atelevision commercial) 308A of the unique identification of compressedmoving image beforehand extracted from other moving image data, astoring unit 110 for storing the retrieving result provided by theretrieving unit 309, and an output unit 111 for displaying and/orprinting the retrieving result stored in the storing unit 110.

The MPEG-2 stream (the television broadcasting) 301A is provided byencoding the moving image of the current television broadcasting.

On the apparatus for monitoring television commercials according to thepresent invention constituted like the above, an operation will beexplained in the following.

FIGS. 47A and 47B are flowcharts of the process in the apparatus formonitoring television commercials to the present invention described inFIG. 46. Still, an identical step number is appended to the step havingthe identical process in FIGS. 47A and 47B with FIGS. 4A and 4B. InFIGS. 46, 47A and 47B, at first the extracting unit 302A for extractingthe DC component fetches the MPEG-2 stream (the television broadcasting)301A to be a retrieval key for identifying the compressed moving image(in the step 401A). Then, the extracting unit 302A extracts theinformation (the DC component) 303A of the feature quantity of thecompressed moving image, which includes the only DC components of theDCT coefficients of each frame, from the MPEG-2 stream 301A (in the step402A).

Next, the extracting unit (the calculating unit for calculating theaverage color information) 304A for extracting the information of theunique identification of the compressed moving image calculates anaverage value by using the DC components of each frame, which isincluded in the information 303A. Then, the extracting unit 304Aarranges the average values on the time axis and extracts thearrangement of the average values on the time axis as the information(the time series information of the average color information) 305A ofthe unique identification of the compressed moving image to be used foridentifying the compressed moving image (in the step 403A).

The storing unit 306 for storing the information of the uniqueidentification of the compressed moving image stores the information305A extracted by the extracting unit 304A (in the step 404).

In the twenty-third embodiment, the information (the time seriesinformation of the average color information) 305A of the uniqueidentification of the compressed moving image consists of three kinds oftime series information, which are information of luminance and colordifference (Y, Cb, Cr). By making a graph of each time seriesinformation, which it makes the horizontal line to be time and makes thevertical line to be a value of the average color information, theinformation 305A is shown by three graphs.

On the other hand, the information (the time series information of theaverage color information of the television commercial) 308A of theunique identification of the compressed moving image is extracted fromthe MPEG-2 stream of the television commercial (the object compressedmoving image data for retrieving) (in the step 405). This extractingprocess can be executed by the processing equal to the foregoing steps401A to 404.

The MPEG-2 stream of the television commercial is provided by encodingthe current television broadcasting equal to the above mentioned MPEG-2stream 301A.

Then, the retrieving unit 309 for retrieving the information of theunique identification of the compressed moving image compares theinformation 305A of the unique identification of the compressed movingimage stored in the storing unit 306 with the information 308A of theunique identification of the compressed moving image extracted from theother moving image data (in the step 406).

The apparatus for retrieving the moving image of the twenty-thirdembodiment is considered that is realized using the simple facility asPC (Personal Computer). Consequently, in the present televisionbroadcasting, ghost phenomenon, in which image is multiply reflected byreflection and multipath of the television radio wave, and colorslippage and the like state arise. A little effect is also generated inthe information 305A extracted using the television broadcasting.

As mentioned above, the information 305A is shown by 3 graphs, andeffect is also generated in the graph. However, since there issufficient information of unique identification of moving image to beused for identifying the moving image in the graph, it is possible to bedifficult to receive the effect by expanding and reducing the graph.

This comparing result is stored in the storing unit 110 for storing theretrieving result (in the step 207). In the case of displaying and/orprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (inthe step 209).

The process of the above mentioned steps 406 to 209 is executed for allof the information 308A of the unique identification of the compressedmoving image extracted from the other moving image data (in the step410).

The apparatus and the method for retrieving the moving image accordingto the invention are explained the above.

The experimental system shown in FIG. 46 was mounted, and the experimenttesting the effectiveness of this invention was executed. In thefollowing, the result of the experiment is shown.

In the experiment for retrieving the television commercials that are 152in number and 37 in type from the television broadcasting of 26 hours 15minutes which is ground wave television broadcasting, the number ofomission was 0 and the number of over-detection was 0.

In the above mentioned methods and apparatuses for retrieving the movingimage, which are referred from in the first embodiment to in thetwenty-third embodiment, a process for dividing an image into aplurality of frames is executed. It is also possible to execute aprocess for dividing each of the frames into “n” (“n” is a naturalnumber 2 or over) regions in addition to the above mentioned process. Inthis case, the information of the unique identification of the movingimage and/or the information of the unique identification of thecompressed moving image for identifying the moving image are extractedrespectively from the divided regions, and then, the extractedinformation is applied to the information 105 of the uniqueidentification of the moving image and/or the information 305 and 305Aof the unique identification of the compressed moving image. And, aretrieving unit 109 for retrieving the information of the uniqueidentification of the moving image and/or a retrieving unit 309 forretrieving the information of the unique identification of thecompressed moving image retrieve the identical information with theinformation 105 and or the information 305 and 305A from the information108 of the unique identification of the moving image and/or theinformation 308 and 308A of the unique identification of the compressedmoving image beforehand extracted. In this retrieving process, theidentical information with the information 105 and/or the information305 and 305A may be extracted with corresponding in all regions of theinformation 105 and/or the information 305 and 305A. Like the above, byretrieving a frame with a plurality of information values, the precisionof the retrieving process may be further improved.

And more, by dividing each of the frames into “n” regions and extractingthe information 105 and/or the information 305 and 305A from each of theregions, it is possible to shorten time series information included inthe information 105 and/or the information 305 and 305A.

Moreover, when the information 105 and/or the information 305 and 305Aare extracted from “n” regions of each of the frames and a process ofextracting moving image is executed by using the information 105 and/orthe information 305 and 305A of limited regions over “n-m” (1≦m≦(n-1)),it is possible to retrieve similar moving image with the difference in apart of regions. That is to say, the retrieving process may be executedwith corresponding in the part of the regions selected as objects forretrieving, consequently, it is possible to retrieve similar movingimage.

Until now, the methods and the apparatuses for retrieving the movingimage, which are referred from in the first embodiment to in thetwenty-third embodiment, are explained. A program for executingprocesses of the above mentioned methods for retrieving the moving imagemay be executed by a computer system.

Still, in the above mentioned embodiments according to the invention,although either the uncompressed moving image data or the compressedmoving image data was applied as the object moving image data forretrieving, both the uncompressed moving image data and the compressedmoving image data may be applied as the object moving image data forretrieving.

And, in the above mentioned embodiments according to the invention, theinformation 105 or 305 was extracted by using over one information fromeach of the frames. However, the information 105 or 305 will beextracted by using over two information from each of the frames,consequently, the precision of the retrieving process may be improved.

In addition, when the information 105 or 305 is extracted by using overtwo information from each frame, the retrieving unit 109 or 309 mayretrieve similar moving image according to comparing the information 105or 305 by using a part of the information values from each of theframes.

Although the invention has been described in its preferred form with acertain degree of particularity, it is understood that the presentdisclosure of the preferred form has been changed in the details ofconstruction and the combination and arrangement of parts may beresorted to without departing from the spirit and the scope of theinvention as hereinafter claimed.

1-57. (canceled)
 58. An apparatus for retrieving a moving image toretrieve a pre-determined moving image from an object moving image,comprising: first extracting means for extracting information of thefeature quantity of each frame from the pre-determined moving image;second extracting means for extracting information of a uniqueidentification created by arranging one or more physical quantities ofthe information of the feature quantity on a time axis; and retrievingmeans for retrieving a position of the pre-determined moving image onthe time axis from the object moving image by comparing the arrangementof the information of the unique identification of the pre-determinedmoving image, with that of the information of the unique identificationprepared in accordance with the object moving image.
 59. The apparatusfor retrieving the moving image of claim 58, further comprising:detecting means for detecting cut points of the moving image from thepre-determined moving image and dividing the pre-determined moving imageinto a plurality of shots at each of the cut points, wherein: the secondextracting means extracts the information of the unique identificationcreated by arranging one or more physical quantities of the informationof the feature quantity at each of the shots divided by the detectingmeans; and the retrieving means retrieves the pre-determined movingimage from the object moving image by comparing the arrangement of theinformation of the unique identification of the pre-determined movingimage, with that of the information of the unique identificationprepared in accordance with the object moving image.
 60. The apparatusfor retrieving the moving image of claim 58, wherein the secondextracting means extracts the information of the unique identificationcreated by arranging one or more expanded or reduced physical quantitiesof the information of the physical quantity on a time axis.
 61. Theapparatus for retrieving the moving image of claim 58, wherein the firstextracting means extracts the information as a block unit of M×N pixels,where M and N are natural numbers, from the pre-determined moving imagedata, as the information of the feature quantity.
 62. The apparatus forretrieving the moving image of claim 58, wherein one or more physicalquantities of the information of the feature quantity are arranged onthe time axis making a horizontal line of a graph represent time. 63.The apparatus for retrieving the moving image of claim 58, wherein theinformation of the unique identification prepared in accordance with theobject moving image is a certain time duration.
 64. The apparatus forretrieving the moving image of claim 58, wherein the retrieving meanscompares an expanded or reduced value of the information of the uniqueidentification which is extracted from the pre-determined moving image,with an expanded or reduced value of the information of the uniqueidentification which is prepared in accordance with the object movingimage, on the time axis.
 65. The apparatus for retrieving the movingimage of claim 58, wherein the second extracting means extracts a valueas the information of the unique identification of pre-determined movingimage by arranging all information of the feature quantity, or someinformation of the feature quantity extracted by uniform or irregularintervals, on the time axis, from the pre-determined moving image. 66.The apparatus for retrieving the moving image of claim 58, wherein thesecond extracting means extracts a value as the information of theunique identification of object moving image prepared in accordance withthe object image by arranging all information of the feature quantity,or some information of the feature quantity extracted by uniform orirregular intervals, on the time axis, from the object moving image. 67.The apparatus for retrieving the moving image of claim 58, wherein theinformation of the feature quantity of each frame extracted by the firstextracting means is physical information of the feature quantityobtained from information of luminance and color difference (Y, Cb, Cr)or information of color (R, G, B) of each frame.
 68. The apparatus forretrieving the moving image of claim 67, wherein the information of thefeature quantity of each frame extracted by the first extracting meansis one or more transformed values obtained from the information ofluminance and color difference (Y, Cb, Cr) or the information of color(R, G, B) of each frame.
 69. The apparatus for retrieving the movingimage of claim 68, wherein the transformed value obtained from theinformation of luminance and color difference (Y, Cb, Cr) of each frameis one or more of a value, a mean value, a total value, and a differencevalue of the physical information of the feature quantity obtained fromthe information of luminance and color difference (Y, Cb, Cr) of eachframe; and the transformed value obtained from the information of color(R, G, B) of each frame is one or more of a value, a mean value, a totalvalue, and a difference value of the physical information of the featurequantity obtained from the information of color (R, G, B) of each frame.70. The apparatus for retrieving the moving image of claim 68, whereinthe transformed value obtained from the information of luminance andcolor difference (Y, Cb, Cr) of each frame is one or more of a value, amean value, a total value, and a difference value of weighted physicalinformation of the feature quantity obtained from the information ofluminance and color difference (Y, Cb, Cr) of each frame; and thetransformed value obtained from the information of color (R, G, B) ofeach frame is one or more of the value, the mean value, the total value,and the difference value of the weighted physical information of thefeature quantity obtained from the information of color (R, G, B) ofeach frame.
 71. The apparatus for retrieving the moving image of claim69, wherein the transformed value obtained from the information ofluminance and color difference (Y, Cb, Cr) of each frame is a quantifiedvalue of the physical information of the feature quantity obtained fromthe information of luminance and color difference (Y, Cb, Cr) of eachframe, or a quantified value obtained by transforming the physicalinformation of the feature quantity from the information of luminanceand color difference (Y, Cb, Cr) of each frame; and the transformedvalue obtained from the information of the information of color (R, G,B) of each frame is a quantified value of the physical information ofthe feature quantity obtained from the information of the information ofcolor (R, G, B) of each frame, or a quantified value obtained bytransforming the physical information of the feature quantity from theinformation of the information of color (R, G, B) of each frame.
 72. Theapparatus for retrieving the moving image of claim 58, wherein the firstextracting means extracts information of a feature quantity of acompressed moving image of each frame by executing a Discrete CosineTransform operation or from a coded stream.
 73. The apparatus forretrieving the moving image of claim 72, wherein the first extractingmeans extracts Discrete Cosine Transform coefficients of each frame orinformation of motion compensation of each frame, as the information ofthe feature quantity of the compressed moving image.
 74. The apparatusfor retrieving the moving image of claim 73, wherein the information ofthe feature quantity of the compressed moving image of each frameextracted by the first extracting means is one of a value, a mean value,a total value, and a difference value of physical information of thefeature quantity of the compressed moving image obtained from theDiscrete Cosine Transform coefficients of each frame, and a vector or avalue of the vector which is at least one of a motion vector, a meanmotion vector between frames before and after, a total motion vector, adifference motion vector, and a motion vector on a whole frame obtainedfrom the information of the motion compensation.
 75. The apparatus forretrieving the moving image of claim 73, wherein the information of thefeature quantity of the compressed moving image of each frame extractedby the first extracting means is one of a value, a mean value, a totalvalue, and a difference value of weighted physical information of thefeature quantity of the compressed moving image obtained from theDiscrete Cosine Transform coefficients of each frame, and a vector or avalue of the vector which is at least one of a motion vector, a meanmotion vector between frames before and after, a total motion vector, adifference motion vector, and a motion vector on a whole frame obtainedfrom weighted information of the motion compensation.
 76. The apparatusfor retrieving the moving image of claim 72, wherein the firstextracting means extracts only a direct current element of the DiscreteCosine Transform coefficients of each frame, as the information of thefeature quantity of the compressed moving image.
 77. The apparatus forretrieving the moving image of claim 73, wherein the information of thefeature quantity of the compressed moving image of each frame extractedby the first extracting means is a quantified value of one of theDiscrete Cosine Transform coefficients, a value of the motioncompensation, and a value obtained from the value of the motioncompensation.
 78. The apparatus for retrieving the moving image of claim58, wherein the information of the feature quantity of the compressedmoving image of each frame extracted by the first extracting means isthe combination of physical information of the feature quantity of eachframe which is obtained from information of luminance and colordifference (Y, Cb, Cr) or information of color (R, G, B) of each frame,and information of a feature quantity of a compressed moving image whichis obtained by executing a Discrete Cosine Transform operation or byextracting from a coded stream.
 79. An apparatus for monitoringtelevision commercials to retrieve a pre-determined moving image such asa television commercial from an object moving image such as thetelevision broadcast and a stored moving image, comprising: firstextracting means for extracting information of a feature quantity ofeach frame from the pre-determined moving image; second extracting meansfor extracting information of a unique identification created byarranging one or more physical quantities of the information of thefeature quantity on a time axis; retrieving means for retrieving aposition of the pre-determined moving image on the time axis from theobject moving image by comparing the arrangement of the information ofthe unique identification of the pre-determined moving image, with thatof the information of the unique identification prepared in accordancewith the object moving image; and displaying means for displaying aresult of retrieving by the retrieving means.
 80. The apparatus formonitoring television commercials of claim 79, further comprisingstoring means for storing the result of retrieving by the retrievingmeans, wherein the displaying means displays the stored result stored bythe storing means.
 81. A method for retrieving a moving image toretrieve a pre-determined moving image from an object moving image,comprising: (A) extracting information of a feature quantity of eachframe from the pre-determined moving image; (B) extracting informationof a unique identification created by arranging one or more physicalquantities of the information of the feature quantity on a time axis;and (C) retrieving a position of the pre-determined moving image on thetime axis from the object moving image by comparing the arrangement ofthe information of the unique identification of the pre-determinedmoving image, with that of the information of the unique identificationprepared in accordance with the object moving image.
 82. The method forretrieving the moving image of claim 81, further comprising: (D)detecting cut points of the moving image from the pre-determined movingimage and dividing the pre-determined moving image into a plurality ofshots at each of the cut points, wherein: the step (B) extracts theinformation of the unique identification created by arranging one ormore physical quantities of the information of the feature quantity ateach of the shots divided by the step (D); and the step (C) retrievesthe pre-determined moving image from the object moving image bycomparing the arrangement of the information of the uniqueidentification of the pre-determined moving image, with that of theinformation of the unique identification prepared in accordance with theobject moving image.
 83. The method for retrieving the moving image ofclaim 81, wherein the step (B) extracts the information of the uniqueidentification created by arranging one or more expanded or reducedphysical quantities of the information of the physical quantity on atime axis.
 84. The method for retrieving the moving image of claim 81,wherein the step (A) extracts the information as a block unit of M×Npixels, where M and N are natural numbers, from the pre-determinedmoving image data, as the information of the feature quantity.
 85. Themethod for retrieving the moving image of claim 81, wherein one or morephysical quantities of the information of the feature quantity arearranged on the time axis making a horizontal line of a graph representtime.
 86. The method for retrieving the moving image of claim 81,wherein the information of the unique identification prepared inaccordance with the object moving image is a certain time duration. 87.The method for retrieving the moving image of claim 81, wherein the step(C) compares an expanded or reduced value of the information of theunique identification which is extracted from the pre-determined movingimage, with an expanded or reduced value of the information of theunique identification which is prepared in accordance with the objectmoving image, on the time axis.
 88. The method for retrieving the movingimage of claim 81, wherein the step (B) extracts a value as theinformation of the unique identification of pre-determined moving imageby arranging all information of the feature quantity, or someinformation of the feature quantity extracted by uniform or irregularintervals, on the time axis, from the pre-determined moving image. 89.The method for retrieving the moving image of claim 81, wherein the step(B) extracts a value as the information of the unique identification ofthe object moving image prepared in accordance with the object image byarranging all information of the feature quantity, or some informationof the feature quantity extracted by uniform or irregular intervals, onthe time axis, from the object moving image.
 90. The method forretrieving the moving image of claim 81, wherein the information of thefeature quantity of each frame extracted by the step (A) is physicalinformation of the feature quantity obtained from information ofluminance and color difference (Y, Cb, Cr) or the information of color(R, G, B) of each frame.
 91. The method for retrieving the moving imageof claim 90, wherein the information of the feature quantity of eachframe extracted by the step (A) is one or more transformed valuesobtained from the information of luminance and color difference (Y, Cb,Cr) or the information of color (R, G, B) of each frame.
 92. The methodfor retrieving the moving image of claim 91, wherein the transformedvalue obtained from the information of luminance and color difference(Y, Cb, Cr) of each frame is one or more of a value, a mean value, atotal value, and a difference value of the physical information of thefeature quantity obtained from the information of luminance and colordifference (Y, Cb, Cr) of each frame; and the transformed value obtainedfrom the information of color (R, G, B) of each frame is one or more ofa value, a mean value, a total value, and a difference value of thephysical information of the feature quantity obtained from theinformation of color (R, G, B) of each frame.
 93. The method forretrieving the moving image of claim 91, wherein the transformed valueobtained from the information of luminance and color difference (Y, Cb,Cr) of each frame is one or more of a value, a mean value, a totalvalue, and a difference value of weighted physical information of thefeature quantity obtained from the information of luminance and colordifference (Y, Cb, Cr) of each frame; and the transformed value obtainedfrom the information of color (R, G, B) of each frame is one or more ofa value, a mean value, a total value, and a difference value of weightedphysical information of the feature quantity obtained from theinformation of color (R, G, B) of each frame.
 94. The method forretrieving the moving image of claim 92, wherein the transformed valueobtained from the information of luminance and color difference (Y, Cb,Cr) of each frame is a quantified value of the physical information ofthe feature quantity obtained from the information of luminance andcolor difference (Y, Cb, Cr) of each frame, or a quantified valueobtained by transforming the physical information of the featurequantity from the information of luminance and color difference (Y, Cb,Cr) of each frame; and the transformed value obtained from theinformation of the information of color (R, G, B) of each frame is aquantified value of the physical information of the feature quantityobtained from the information of color (R, G, B) of each frame, or aquantified value obtained by transforming the physical information ofthe feature quantity from the information of the information of color(R, G, B) of each frame.
 95. The method for retrieving the moving imageof claim 81, wherein step (A) extracts the information of the featurequantity of the compressed moving image of each frame by executing adiscrete Cosine Transform operation or extracting from a coded stream.96. The method for retrieving the moving image of claim 95, wherein step(A) extracts the Discrete Cosine Transform coefficients of each frame orthe information of motion compensation of each frame, as the informationof the feature quantity of compressed moving image.
 97. The method forretrieving the moving image of claim 96, wherein the information of thefeature quantity of compressed moving image of each frame extracted bythe step (A) is one of a value, a mean value, a total value, and adifference value of the physical information of the feature quantity ofthe compressed moving image obtained from the Discrete Cosine Transformcoefficients of each frame, and a vector or a value of the vector whichis at least one of a motion vector, a mean motion vector between framesbefore and after, a total motion vector, a difference motion vector, anda motion vector on a whole frame obtained from the information of themotion compensation.
 98. The method for retrieving the moving image ofclaim 96, wherein the information of the feature quantity of compressedmoving image of each frame extracted by step (A) is one of a value, amean value, a total value, and a difference value of weighted physicalinformation of the feature quantity of the compressed moving imageobtained from Discrete Cosine Transform coefficients of each frame, anda vector or a value of the vector which is at least one of a motionvector, a mean motion vector between frames before and after, a totalmotion vector, a difference motion vector, and a motion vector on awhole frame obtained from the weighted information of the motioncompensation.
 99. The method for retrieving the moving image of claim95, wherein step (A) extracts only a Direct Current element of theDiscrete Cosine Transform coefficients of each frame, as the informationof the feature quantity of the compressed moving image.
 100. The methodfor retrieving the moving image of claim 96, wherein the information ofthe feature quantity of the compressed moving image of each frameextracted by step (A) is a quantified value of one of the DiscreteCosine Transform coefficients, a value of the motion compensation, and avalue obtained from the value of the motion compensation.
 101. Themethod for retrieving the moving image of claim 81, wherein theinformation of the feature quantity of the compressed moving image ofeach frame extracted by the step (A) is the combination of the physicalinformation of the feature quantity of each frame which is obtained frominformation of luminance and color difference (Y, Cb, Cr) or informationof color (R, G, B) of each frame, and information of a feature quantityof a compressed moving image which is obtained by executing a DiscreteCosine Transform operation or by extracting from a coded stream.
 102. Amethod for monitoring television commercials to retrieve apre-determined moving image such as a television commercial from anobject moving image such as a television broadcast and stored movingimage, comprising: (A) extracting information of a feature quantity ofeach frame from the pre-determined moving image; (B) extractinginformation of a unique identification created by arranging one or morephysical quantities of the information of the feature quantity on a timeaxis; (C) retrieving a position of the pre-determined moving image onthe time axis from the object moving image by comparing the arrangementof the information of the unique identification of the pre-determinedmoving image, with that of the information of the unique identificationprepared in accordance with the object moving image; and (E) displayingthe result of retrieving by step (C).
 103. The method for monitoringtelevision commercials of claim 102, further comprising a step (F) ofstoring the result of retrieving by step (C), wherein step(E) displaysthe stored result stored by step (F).
 104. Program products forretrieving a moving image to retrieve a pre-determined moving image froman object moving image, which is executed by a computer system,comprising: (A) extracting information of a feature quantity of eachframe from the pre-determined moving image; (B) extracting informationof a unique identification created by arranging one or more physicalquantities of the information of the feature quantity on a time axis;and (C) retrieving a position of the pre-determined moving image on thetime axis from the object moving image by comparing the arrangement ofthe information of the unique identification of the pre-determinedmoving image with that of the information of the unique identificationprepared in accordance with the object moving image.
 105. Programproducts for monitoring television commercials to retrieve apre-determined moving image such as a television commercial from anobject moving image such as the television broadcast and a stored movingimage, which is executed by computer system, comprising: (A) extractinginformation of a feature quantity of each frame from the pre-determinedmoving image; (B) extracting information of a unique identificationcreated by arranging one or more physical quantities of the informationof the feature quantity on a time axis; (C) retrieving a position of thepre-determined moving image on the time axis from the object movingimage by comparing the arrangement of the information of the uniqueidentification of the pre-determined moving image, with that of theinformation of the unique identification prepared in accordance with theobject moving image; and (E) displaying the result of retrieving by step(C).